Return-Path: X-Original-To: apmail-crunch-dev-archive@www.apache.org Delivered-To: apmail-crunch-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1CC781103F for ; Fri, 4 Jul 2014 15:30:35 +0000 (UTC) Received: (qmail 15249 invoked by uid 500); 4 Jul 2014 15:30:35 -0000 Delivered-To: apmail-crunch-dev-archive@crunch.apache.org Received: (qmail 15213 invoked by uid 500); 4 Jul 2014 15:30:35 -0000 Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@crunch.apache.org Delivered-To: mailing list dev@crunch.apache.org Received: (qmail 15179 invoked by uid 500); 4 Jul 2014 15:30:34 -0000 Delivered-To: apmail-incubator-crunch-dev@incubator.apache.org Received: (qmail 15170 invoked by uid 99); 4 Jul 2014 15:30:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jul 2014 15:30:34 +0000 Date: Fri, 4 Jul 2014 15:30:34 +0000 (UTC) From: "Gabriel Reid (JIRA)" To: crunch-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (CRUNCH-433) Add support for reading specific/reflect data from an Avro MR file MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Gabriel Reid created CRUNCH-433: ----------------------------------- Summary: Add support for reading specific/reflect data from an Avro MR file Key: CRUNCH-433 URL: https://issues.apache.org/jira/browse/CRUNCH-433 Project: Crunch Issue Type: New Feature Reporter: Gabriel Reid Assignee: Gabriel Reid An Avro Key/Value file written via raw MapReduce contains records that follow the schema generated by the org.apache.avro.hadoop.io.AvroKeyValue class. If these files contain specific or reflection-based records, there is currently no easy way to read them in as specific or reflection records. Using the basic public Crunch APIs, they can only be read as generic records (that also contain generic records). A method should be added to the Avros class which allows specifying specific PTypes to be used for reading the underlying data types within a raw MR output file. Link to related discussion that inspired this ticket on the user list: http://s.apache.org/es -- This message was sent by Atlassian JIRA (v6.2#6252)