crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (JIRA)" <j...@apache.org>
Subject [jira] [Created] (CRUNCH-433) Add support for reading specific/reflect data from an Avro MR file
Date Fri, 04 Jul 2014 15:30:34 GMT
Gabriel Reid created CRUNCH-433:
-----------------------------------

             Summary: Add support for reading specific/reflect data from an Avro MR file
                 Key: CRUNCH-433
                 URL: https://issues.apache.org/jira/browse/CRUNCH-433
             Project: Crunch
          Issue Type: New Feature
            Reporter: Gabriel Reid
            Assignee: Gabriel Reid


An Avro Key/Value file written via raw MapReduce contains records that follow the schema generated
by the org.apache.avro.hadoop.io.AvroKeyValue class. 

If these files contain specific or reflection-based records, there is currently no easy way
to read them in as specific or reflection records. Using the basic public Crunch APIs, they
can only be read as generic records (that also contain generic records).

A method should be added to the Avros class which allows specifying specific PTypes to be
used for reading the underlying data types within a raw MR output file.

Link to related discussion that inspired this ticket on the user list: http://s.apache.org/es



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message