hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Kimball (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1480) CombineFileRecordReader does not properly initialize child RecordReader
Date Fri, 05 Mar 2010 01:29:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12841641#action_12841641

Aaron Kimball commented on MAPREDUCE-1480:


Thanks for looking over this patch.

The previous progress calculator was strictly based on the number of sub-splits processed.
The underlying RecordReader's getProgress() function was never called, which means that the
granularity of progress was only based around the number of subsplits and did not take intra-split
progress into account. A review from Dhruba is definitely welcome.

I'll add another testcase as you suggest and post this in the next couple of days.

> CombineFileRecordReader does not properly initialize child RecordReader
> -----------------------------------------------------------------------
>                 Key: MAPREDUCE-1480
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1480
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1480.2.patch, MAPREDUCE-1480.patch
> CombineFileRecordReader instantiates child RecordReader instances but never calls their
initialize() method to give them the proper TaskAttemptContext.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message