Amazing technological breakthrough possible @S-Logix pro@slogix.in

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • pro@slogix.in
  • +91- 81240 01111

Social List

How to Initialize and Stop Spark Session in R?

Description

To Initialize Spark Session in R

Process
  • Set up spark home
  • Load the spark library
  • Initialize the spark context
  • Now write the desired coding
  • Stop the spark session
Sapmle Code

#Set up spark home
Sys.setenv(SPARK_HOME=”/…/spark-2.4.0-bin-hadoop2.7″)
.libPaths(c(file.path(Sys.getenv(“SPARK_HOME”), “R”, “lib”), .libPaths()))
#Load the library
library(SparkR)
#Initialize the Spark Context
#To run spark in a local node give master=”local”
sc #Start the SparkSQL Context
sqlContext #To stop the spark session
sparkR.session.stop()

Screenshots
How to Initialize and Stop Spark Session in R
List