incubator-hcatalog-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Travis Crawford (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HCATALOG-373) ProgressReporter should work with both old and new MR API
Date Thu, 26 Apr 2012 00:47:16 GMT

     [ https://issues.apache.org/jira/browse/HCATALOG-373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Travis Crawford updated HCATALOG-373:
-------------------------------------

    Attachment: HCATALOG-373_progress_reporter_3.diff

I believe this version of the patch is ready to go.

Please see my previous comment for a description of why creation of the base record reader
is delayed until initialized. 

ProgressReporter is updated to work in both old and new API contexts, which is sometimes required.
In the particular case that caused me to look at this, the Elephant-Bird deprecated API wrappers
were having issues with this reporter.
                
> ProgressReporter should work with both old and new MR API
> ---------------------------------------------------------
>
>                 Key: HCATALOG-373
>                 URL: https://issues.apache.org/jira/browse/HCATALOG-373
>             Project: HCatalog
>          Issue Type: Bug
>            Reporter: Travis Crawford
>            Assignee: Travis Crawford
>         Attachments: HCATALOG-373_progress_reporter.diff, HCATALOG-373_progress_reporter_2.diff,
HCATALOG-373_progress_reporter_3.diff
>
>
> {{org.apache.hcatalog.mapreduce.ProgressReporter}} currently implements {{org.apache.hadoop.mapred.Reporter}}.
It should also extend {{org.apache.hadoop.mapreduce.StatusReporter}} so it works with code
expecting either an old or new API reporter.
> The use case is using a wrapper so a serde works with a new-API input format.
> https://github.com/kevinweil/elephant-bird/blob/master/src/java/com/twitter/elephantbird/mapred/input/DeprecatedInputFormatWrapper.java#L163

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message