Mailing-List: contact dev-help@crunch.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@crunch.apache.org
Date: Mon, 29 Apr 2013 03:52:16 +0000 (UTC)
From: "Josh Wills (JIRA)" <jira@apache.org>
To: crunch-dev@incubator.apache.org
Message-ID: <JIRA.12644930.1367008869932.240428.1367207536529@arcas>
In-Reply-To: <JIRA.12644930.1367008869932@arcas>
References: <JIRA.12644930.1367008869932@arcas>
Subject: [jira] [Commented] (CRUNCH-199) Add Source and Target Support for
 AvroTrevniKey files
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CRUNCH-199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644246#comment-13644246 ] 

Josh Wills commented on CRUNCH-199:
-----------------------------------

Hey Micah-- the thing that moves the data from the temporary location that is created during the crunch job run to the final location is the MSCROutputHandler in o.a.c.impl.mr.plan. I think the source of the issue is at the end of the configureMapReduce function in the TrevniKeyTarget, where you reset the outputPath. I think that this line should go.

Also, I agree with your comment: the TrevniKeyOutputFormat isn't aware of multiple schema outputs within the context of a single job, so you'll need to use a FormatBundle directly so that different config schemas don't conflict with each other. I'll post a patch to demonstrate.


> Add Source and Target Support for AvroTrevniKey files
> -----------------------------------------------------
>
>                 Key: CRUNCH-199
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-199
>             Project: Crunch
>          Issue Type: Wish
>          Components: Core, IO
>            Reporter: Micah Whitacre
>            Assignee: Josh Wills
>            Priority: Minor
>         Attachments: CRUNCH-199.patch
>
>
> Add Source, Target, and SourceTarget, support for AvroTrevniKeyInputFormat[1] and AvroTrevniKeyOutputFormat[2].
> [1] - http://avro.apache.org/docs/1.7.4/api/java/org/apache/trevni/avro/mapreduce/AvroTrevniKeyInputFormat.html
> [2] - http://avro.apache.org/docs/1.7.4/api/java/org/apache/trevni/avro/mapreduce/AvroTrevniKeyOutputFormat.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira