incubator-crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <jwi...@cloudera.com>
Subject Re: Saving collection to text files in Scrunch
Date Thu, 29 Nov 2012 16:03:56 GMT
Hey Roman,

While I take a look at that, would you try using the writeTextFile function
(e.g., writeTextFile(<pcollection>, args(1)) ) and let me know if that does
the trick?

Josh


On Thu, Nov 29, 2012 at 6:34 AM, Roman V. Shapovalov <
shapovalov@graphics.cs.msu.su> wrote:

> Dear crunch-users,
>
> I am trying to solve some toy MapReduce problem using Scrunch. When I
> write the final result in the pipeline app, i.e. call
>
> write(to.textFile(args(1)))
>
> and get object names in the output file, like:
>
> org.apache.avro.mapred.AvroWrapper@80
> org.apache.avro.mapred.AvroWrapper@17a73
>
> This happens only if I perform some mapping (even identity); just
> reading and writing results in good strings in the file.
>
> It seems that mapping wraps the strings using the AvroWrapper, but
> writing to the text file does not unwrap them. Is it supposed to
> unwrap them?
>
> There is a factory method To.formattedFile() in Crunch (I guess it may
> help, but it is not documented), but it is not ported to Scrunch. Is
> there another idiom for writing strings?
>
> Thanks in advance,
> Roman
>



-- 
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>

Mime
View raw message