incubator-crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: Facing error java.io.NotSerializableException: org.apache.hadoop.io.Text
Date Wed, 26 Dec 2012 13:34:53 GMT
Hi Ashish,

Are you holding on to a non-transient Text instance in a DoFn perhaps? DoFns need to remain
serializable. 

Otherwise, could you post your (non-working) code (I'm assuming its pretty short). 

- Gabriel

On 26 Dec 2012, at 13:54, Ashish <paliwalashish@gmail.com> wrote:

> Folks,
> 
> I was trying to port Word co-occurrence example(using Pairs) to Crunch. Had used famous
TextPair class from Hadoop Definitive Guide.
> While running getting this error
> 
> ERROR mr.MRPipeline: org.apache.crunch.impl.mr.run.CrunchRuntimeException: java.io.NotSerializableException:
org.apache.hadoop.io.Text
> 
> As an alternative, I created WordPair class that uses String instead of Text and implemented
Serializable, WritableComparable. This piece worked.
> 
> Is this behavior expected or I am missing something?
> 
> 
> -- 
> thanks
> ashish
> 
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal

Mime
View raw message