WordCount

Description

WordCount
Nilesh Patel
Note by Nilesh Patel, updated more than 1 year ago
Nilesh Patel
Created by Nilesh Patel over 6 years ago
4
0

Resource summary

Page 1

Word Count:   Example 1 (RDD) val txtfile=sc.textFile("C:/Users/Redirection/pateln7/Downloads/SPARK/wordcount.txt") val txtflatmap= txtfile.flatMap(x=>x.split(" ")) txtflatmap.first val txtmap=txtflatmap.map(x=>(x,1)) txtmap.first val output=txtmap.reduceByKey(_+_) output.foreach(println)   Single line : val output = txtfile.flatMap(x=>x.split(" ")).map(x=>(x,1)).reduceByKey(_+_) output.foreach(println)     Example 2  (RDD) val txtfile=sc.textFile("C:/Users/Redirection/pateln7/Downloads/SPARK/wordcount.txt") val txtflatmap= txtfile.flatMap(x=>x.split(" ")) txtflatmap.first val txtmap=txtflatmap.map(x=>(x,1)) txtmap.first val output = txtmap.reduceByKey((x,txtmap)=>(x+txtmap)).foreach(println)   Example 3 (for Array[(string ,Int) ] type data  val str = "Hell saying hello is good, keep say hello"  val x =str.split(" ").map(x=> (x, 1))  val q = sc.parallelize(x)  q.reduceByKey(_+_).foreach(println)

Show full summary Hide full summary

Similar

Filter and Map
Nilesh Patel
Joins
Nilesh Patel
Setup spark scala in windows
Nilesh Patel
Geography Coastal Zones Flashcards
Zakiya Tabassum
Ebola Virus Disease
rubyduggan
Input, output and storage devices
Mr A Esch
RadioTelefonia
Adriana Forero
SalesForce ADM 201 Study Quiz
Brianne Wright
Část 3.
Gábi Krsková
Macbeth Quotes To Learn
Sophie Brokenshire