Amazing technological breakthrough possible @S-Logix pro@slogix.in

Office Address

  • #5, First Floor, 4th Street Dr. Subbarayan Nagar Kodambakkam, Chennai-600 024 Landmark : Samiyar Madam
  • pro@slogix.in
  • +91- 81240 01111

Social List

How to execute scala application in Spark shell?

  • The code segment given below performs the word count operation using Scala API. The data from the input file is read and map reduce operation is performed over the data to count the words. The output is stored as a file.
  • Code Segment
    scala> val file= sc.textFile(“samplewordcount.txt”)
    scala> val wordcounts = file.flatMap(line => line.split(” “)).map(word => (word, 1)).reduceByKey(_+_);
    scala> wordcounts.toDebugString
    scala> wordcounts.cache()
    scala> wordcounts.saveAsTextFile(“outputfile”)
List