crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <>
Subject [jira] [Commented] (CRUNCH-199) Add Source and Target Support for AvroTrevniKey files
Date Mon, 29 Apr 2013 03:52:16 GMT


Josh Wills commented on CRUNCH-199:

Hey Micah-- the thing that moves the data from the temporary location that is created during
the crunch job run to the final location is the MSCROutputHandler in I
think the source of the issue is at the end of the configureMapReduce function in the TrevniKeyTarget,
where you reset the outputPath. I think that this line should go.

Also, I agree with your comment: the TrevniKeyOutputFormat isn't aware of multiple schema
outputs within the context of a single job, so you'll need to use a FormatBundle directly
so that different config schemas don't conflict with each other. I'll post a patch to demonstrate.

> Add Source and Target Support for AvroTrevniKey files
> -----------------------------------------------------
>                 Key: CRUNCH-199
>                 URL:
>             Project: Crunch
>          Issue Type: Wish
>          Components: Core, IO
>            Reporter: Micah Whitacre
>            Assignee: Josh Wills
>            Priority: Minor
>         Attachments: CRUNCH-199.patch
> Add Source, Target, and SourceTarget, support for AvroTrevniKeyInputFormat[1] and AvroTrevniKeyOutputFormat[2].
> [1] -
> [2] -

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message