crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gabriel Reid <gabriel.r...@gmail.com>
Subject Re: MemPipeline doesn't support writing Avro records via reflection.
Date Wed, 20 Nov 2013 15:25:56 GMT
Hi David,

On Wed, Nov 20, 2013 at 4:00 PM, David Whiting <davidwhiting@gmail.com> wrote:
> It seems that this was half implemented but never finished, but I needed it
> to create some test data to run through a cluster MapReduce test. The
> current implementation correctly reflects the schema, but then uses a
> GenericDatumWriter to try and write the record, causing a
> ClassCastException. The correct way would be to get a ReflectDatumWriter
> from the ReflectDataFactory.
>
> I've attached a tiny patch that should fix it.

Yes, I think there are definitely some issues with the MemPipeline and
Avro handling.

It looks like your patch didn't make through with your mail (I think
all attachments get stripped out by the mailing list). Could you open
a JIRA ticket at https://issues.apache.org/jira/browse/CRUNCH and post
the patch there?

Thanks,

Gabriel

Mime
View raw message