spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From madhu phatak <phatak....@gmail.com>
Subject Re: save a histogram to a file
Date Fri, 23 Jan 2015 09:23:48 GMT
Hi,
histogram method return normal scala types not a RDD. So you will not
have saveAsTextFile.
You can use makeRDD method make a rdd out of the data and saveAsObject file

val hist = a.histogram(10)
val histRDD = sc.makeRDD(hist)
histRDD.saveAsObjectFile(path)


On Fri, Jan 23, 2015 at 5:37 AM, SK <skrishna.id@gmail.com> wrote:

> Hi,
> histogram() returns an object that is a  pair of Arrays. There appears to
> be
> no saveAsTextFile() for this paired object.
>
> Currently I am using the following to save the output to a file:
>
> val hist = a.histogram(10)
>
> val arr1 = sc.parallelize(hist._1).saveAsTextFile("file1")
> val arr2 = sc.parallelize(hist._2).saveAsTextFile("file2")
>
> Is there a simpler way to save the histogram() result to a file?
>
> thanks
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/save-a-histogram-to-a-file-tp21324.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>


-- 
Regards,
Madhukara Phatak
http://www.madhukaraphatak.com

Mime
View raw message