How to implement Frequency pattern mining using Spark with R ?


To implement frequency pattern mining using Spark with R

  • Set up Spark Context and Spark session
  • Load the Data set
  • Convert each row as a single transaction
  • Generate the frequency pattern mining model and fit it
  • Find the frequent item sets
  • Generate the association rule
  • Examine the input items against all the association rules and summarize the consequents as prediction

#Set up spark home
.libPaths(c(file.path(Sys.getenv(“SPARK_HOME”), “R”, “lib”), .libPaths()))
#Load the library
#Initialize the Spark Context
#To run spark in a local node give master=”local”
sc #Start the SparkSQL Context
sqlContext #Load the data set
data = read.df(“file:///…./GsData.txt”,”csv”,header = “False”, schema = structType(structField(“raw_items”, “string”)), na.strings = “NA”)
data showDF(data,truncate=FALSE)
model #To get the frequent item sets
frequent_itemsets showDF(frequent_itemsets)
#To get the association rules
association_rules showDF(association_rules)
# Predict on new data

Leave Comment

Your email address will not be published. Required fields are marked *

clear formSubmit