Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
Received-SPF: pass (nike.apache.org: domain of phatak.dev@gmail.com designates
 209.85.216.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1421971644455-21324.post@n3.nabble.com>
References: <1421971644455-21324.post@n3.nabble.com>
Date: Fri, 23 Jan 2015 14:53:48 +0530
Message-ID: 
 <CANpNmWV3RX-eP=D8Y_OJG+znoE+UaTu+picbVuQcJ+Y98fcGnA@mail.gmail.com>
Subject: Re: save a histogram to a file
From: madhu phatak <phatak.dev@gmail.com>
To: SK <skrishna.id@gmail.com>
Cc: "user@spark.apache.org" <user@spark.apache.org>
Content-Type: multipart/alternative; boundary=001a113ab98899fc21050d4e58a6

--001a113ab98899fc21050d4e58a6
Content-Type: text/plain; charset=UTF-8

Hi,
histogram method return normal scala types not a RDD. So you will not
have saveAsTextFile.
You can use makeRDD method make a rdd out of the data and saveAsObject file

val hist = a.histogram(10)
val histRDD = sc.makeRDD(hist)
histRDD.saveAsObjectFile(path)


On Fri, Jan 23, 2015 at 5:37 AM, SK <skrishna.id@gmail.com> wrote:

> Hi,
> histogram() returns an object that is a  pair of Arrays. There appears to
> be
> no saveAsTextFile() for this paired object.
>
> Currently I am using the following to save the output to a file:
>
> val hist = a.histogram(10)
>
> val arr1 = sc.parallelize(hist._1).saveAsTextFile("file1")
> val arr2 = sc.parallelize(hist._2).saveAsTextFile("file2")
>
> Is there a simpler way to save the histogram() result to a file?
>
> thanks
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/save-a-histogram-to-a-file-tp21324.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>


-- 
Regards,
Madhukara Phatak
http://www.madhukaraphatak.com

--001a113ab98899fc21050d4e58a6
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<div>histogram method return normal scala types not a R=
DD. So you will not have=C2=A0<span style=3D"font-size:13px">saveAsTextFile=
. You can use makeRDD method make a rdd out of the data and saveAsObject fi=
le</span></div><div><br></div><div><span style=3D"font-size:13px">val hist =
=3D a.histogram(10)</span><br style=3D"font-size:13px"></div><div><span sty=
le=3D"font-size:13px">val histRDD =3D sc.makeRDD(hist)</span></div><div><sp=
an style=3D"font-size:13px">histRDD.saveAsObjectFile(path)</span></div><div=
><span style=3D"font-size:13px"><br></span></div></div><div class=3D"gmail_=
extra"><br><div class=3D"gmail_quote">On Fri, Jan 23, 2015 at 5:37 AM, SK <=
span dir=3D"ltr">&lt;<a href=3D"mailto:skrishna.id@gmail.com" target=3D"_bl=
ank">skrishna.id@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gm=
ail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-le=
ft:1ex">Hi,<br>
histogram() returns an object that is a=C2=A0 pair of Arrays. There appears=
 to be<br>
no saveAsTextFile() for this paired object.<br>
<br>
Currently I am using the following to save the output to a file:<br>
<br>
val hist =3D a.histogram(10)<br>
<br>
val arr1 =3D sc.parallelize(hist._1).saveAsTextFile(&quot;file1&quot;)<br>
val arr2 =3D sc.parallelize(hist._2).saveAsTextFile(&quot;file2&quot;)<br>
<br>
Is there a simpler way to save the histogram() result to a file?<br>
<br>
thanks<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://apache-spark-user-list.1001=
560.n3.nabble.com/save-a-histogram-to-a-file-tp21324.html" target=3D"_blank=
">http://apache-spark-user-list.1001560.n3.nabble.com/save-a-histogram-to-a=
-file-tp21324.html</a><br>
Sent from the Apache Spark User List mailing list archive at Nabble.com.<br=
>
<br>
---------------------------------------------------------------------<br>
To unsubscribe, e-mail: <a href=3D"mailto:user-unsubscribe@spark.apache.org=
">user-unsubscribe@spark.apache.org</a><br>
For additional commands, e-mail: <a href=3D"mailto:user-help@spark.apache.o=
rg">user-help@spark.apache.org</a><br>
<br>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div class=
=3D"gmail_signature"><div dir=3D"ltr"><div>Regards,</div><div>Madhukara Pha=
tak</div><div><a href=3D"http://www.madhukaraphatak.com" target=3D"_blank">=
http://www.madhukaraphatak.com</a><br></div></div></div>
</div>

--001a113ab98899fc21050d4e58a6--