crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Wills <>
Subject Re: crunch 0.8.2+6-cdh4.4.0
Date Tue, 28 Jan 2014 22:08:58 GMT
Hey Stephen,

Slightly confused here, question inlined.

On Tue, Jan 28, 2014 at 12:59 PM, Durfey,Stephen

>  This question is specifically about this version maintained by Cloudera.
>  I was looking to update out Crunch version from 0.8.0-cdh4.3.0
> to 0.8.2+6-cdh4.4.0. In the process some of our tests starting failing from
> NullPointerExceptions. I've discovered why these exceptions are happening,
> but I'm having trouble tracking down the where.
>  The exceptions occur when we emit a Pair<POJO, null> that uses an Avro
> PType. Previously this worked just fine, and by the time the CrunchOutputs
> started writing to a sequence file the value would be an instance of
> NullWritable, and it would successfully pull off the output type for
> serialization (in SequenceFile.BlockCompressWriter#append(k, v)). After the
> version change the value when it got down to write to a sequence file was
> 'null', rather than NullWritable.

It's an AvroType that's getting written to a Sequence File? Is that right?

>  Any thoughts?
>  - Stephen
>  CONFIDENTIALITY NOTICE This message and any included attachments are from
> Cerner Corporation and are intended only for the addressee. The information
> contained in this message is confidential and may constitute inside or
> non-public information under international, federal, or state securities
> laws. Unauthorized forwarding, printing, copying, distribution, or use of
> such information is strictly prohibited and may be unlawful. If you are not
> the addressee, please promptly delete this message and notify the sender of
> the delivery error by e-mail or you may call Cerner's corporate offices in
> Kansas City, Missouri, U.S.A at (+1) (816)221-1024.

View raw message