crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Micah Whitacre (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CRUNCH-293) Injection of reader into AvroRecordReader
Date Mon, 18 Nov 2013 13:19:23 GMT

    [ https://issues.apache.org/jira/browse/CRUNCH-293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13825303#comment-13825303
] 

Micah Whitacre commented on CRUNCH-293:
---------------------------------------

The ones I don't know about mainly.  :)  I just want to verify there aren't cases that I overlooked.
 Like I didn't make the change to the AvroDeepCopier to use the factory mostly because I assumed
it was used for already deserialized values where the schema needed to change and therefore
a custom reader wasn't needed.

> Injection of reader into AvroRecordReader
> -----------------------------------------
>
>                 Key: CRUNCH-293
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-293
>             Project: Crunch
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 0.7.0, 0.8.0
>            Reporter: Micah Whitacre
>            Assignee: Micah Whitacre
>         Attachments: CRUNCH-293.patch, CRUNCH-293_v2.patch
>
>
> With CRUNCH-243, I wanted to support injecting custom readers to handle the cases like
passivity between Avro Schema.  The changes made however were not complete as we also need
to be able to inject a reader into the AvroRecordReader which constructs its own SpecificDatumReader.
> We could create a SpecificDataFactory which emulates the ReflectDataFactory.  Or simplify
to a single DataFactory which will create either Reflect/Specific/Generic.  Thoughts?



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message