crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-242) Input/output conversion needs to be controlled by the Source/Target interfaces
Date Tue, 23 Jul 2013 00:14:49 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Wills updated CRUNCH-242:
------------------------------

    Attachment: CRUNCH-242.patch

Patch that updates the Source and Target interfaces to enable them to override the Converter
used by the PType as appropriate.
                
> Input/output conversion needs to be controlled by the Source/Target interfaces
> ------------------------------------------------------------------------------
>
>                 Key: CRUNCH-242
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-242
>             Project: Crunch
>          Issue Type: Bug
>            Reporter: Josh Wills
>         Attachments: CRUNCH-242.patch
>
>
> I was working on adding support for Parquet to Crunch, and ran into the issue that Parquet
always assumes that the value it returns is on the "value" side of the key-value pair of an
InputFormat/OutputFormat. Crunch, for semi-sensible historical reasons, makes this position
dependent on the PTypeFamily (Avro PTypes write to the key, Writable PTypes write to the value).
Since the Parquet InputFormat/OutputFormat treat the two types the same way, we need a way
for the Source and Target implementations to override the default configuration of the PTypes
and choose the right side for the given format.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message