[vc_row][vc_column][vc_column_text]

How to Initialize and Stop Spark Session in R ?

[/vc_column_text][vc_empty_space][vc_column_text]

Description

To Initialize Spark Session in R

[/vc_column_text][vc_empty_space][vc_tta_tabs style=”modern” shape=”square” active_section=”1″][vc_tta_section title=”Process” tab_id=”1540981759723-27683d14-e602″]

  • Set up spark home
  • Load the spark library
  • Initialize the spark context
  • Now write the desired coding
  • Stop the spark session
[/vc_tta_section][vc_tta_section title=”Sample Code” tab_id=”Source-code”][vc_column_text]#Set up spark home
Sys.setenv(SPARK_HOME=”/…/spark-2.4.0-bin-hadoop2.7″)
.libPaths(c(file.path(Sys.getenv(“SPARK_HOME”), “R”, “lib”), .libPaths()))
#Load the library
library(SparkR)
#Initialize the Spark Context
#To run spark in a local node give master=”local”
sc #Start the SparkSQL Context
sqlContext #To stop the spark session
sparkR.session.stop()[/vc_column_text][vc_empty_space][/vc_tta_section][vc_tta_section title=”Screenshots” tab_id=”Screenshots”][vc_single_image image=”90604″ img_size=”full”][vc_empty_space height=”15px”][/vc_tta_section][/vc_tta_tabs][/vc_column][/vc_row][vc_row][vc_column][vc_empty_space][vc_row_inner][vc_column_inner width=”1/3″][vc_column_text]

Previous Sample

[/vc_column_text][/vc_column_inner][vc_column_inner width=”1/3″][vc_column_text]

List

[/vc_column_text][/vc_column_inner][vc_column_inner width=”1/3″][vc_column_text]

Next Sample

[/vc_column_text][/vc_column_inner][/vc_row_inner][vc_empty_space][/vc_column][/vc_row]

Leave Comment

Your email address will not be published. Required fields are marked *

clear formSubmit