spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kali.tummala@gmail.com" <Kali.tumm...@gmail.com>
Subject Re: Create hashmap using two RDD's
Date Sat, 10 Oct 2015 15:31:38 GMT
Got it ..., created hashmap and saved it to file please follow below steps ..

 val QuoteRDD=quotefile.map(x => x.split("\\|")).
      filter(line => line(0).contains("1017")).
      map(x => ((x(5)+x(4)) , (x(5),x(4),x(1) ,
      if (x(15) =="B")
        (
          {if (x(25) == "") x(9)  else x(25)},
         {if (x(37) == "") x(11) else x(37)}
        )
      else if (x(15) =="C" )
        (
           {if (x(24) == "") (x(9))  else x(24)},
           {if (x(30) == "") (x(11)) else x(30)}
        )
      else if (x(15) =="A")
         {(x(9),x(11))}
        )))


    val QuoteHashMap=QuoteRDD.collect().toMap
    val test=QuoteHashMap.values.toSeq
    val test2=sc.parallelize(test.map(x =>
x.toString.replace("(","").replace(")","")))
    test2.saveAsTextFile("C:\\Users\\kalit_000\\Desktop\\mkdata\\test.txt")
    test2.collect().foreach(println)



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Create-hashmap-using-two-RDD-s-tp24996p25014.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message