spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "" <>
Subject Create hashmap using two RDD's
Date Fri, 09 Oct 2015 14:59:33 GMT
Hi all, 

I am trying to create a hashmap using two rdd, but having issues key not
found ....
do I need to convert RDD to list first ?

1) rdd has key data
2) rdd has value data

Key Rdd:-
val => x.split("\\|")).filter(line =>
line(0).contains("1017")).map(x => x(5)+x(4))

Value Rdd:-
val => x.split("\\|")).
                 filter(line => line(0).contains("1017")).
                 map(x =>
(x(5).toString+x(4).toString,x(5).toString,x(4).toString,x(1).toString ,
                 if (x(15).toString =="B") 
                       if (x(25).toString =="") x(9).toString  else
                       if (x(37).toString =="") x(11).toString else
                if (x(15).toString =="C") 
                       if (x(24).toString =="") x(9).toString  else
                       if (x(30).toString =="") x(11).toString else
                 if (x(15).toString =="A") 

Hash Map:-
val quotehash = new HashMap[String,String]
quotehash + quotekey.toString() -> QuoteRDD

key not found: CPHI080000172


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message