spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SK <>
Subject Re: OutofMemoryError when generating output
Date Thu, 28 Aug 2014 19:45:22 GMT
Thanks for the response. I tried to use countByKey. But I am not able to
write the output to console or to a file. Neither collect() nor
saveAsTextFile() work for the Map object that is generated after

val    x = sc.textFile(baseFile)).map { line =>
                    val fields = line.split("\t")
                   (fields(11), fields(6)) // extract (month, user_id)

x.saveAsTextFile(...)  // does not work. generates an error that
saveAstextFile is not defined for Map object

Is there a way to convert the Map object to an object that I can output to
console and to a file?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message