hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1480) CombineFileRecordReader does not properly initialize child RecordReader
Date Mon, 08 Mar 2010 08:43:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12842580#action_12842580
] 

dhruba borthakur commented on MAPREDUCE-1480:
---------------------------------------------

Actually, while calculating subprogress, it shoudl take into account the size of the current
split:

{code}
  public float getProgress() throws IOException {
    long subprogress = 0;    // bytes processed in current split
    if (null != curReader) {
      subprogress = (long)(curReader.getProgress() * split.getLength(idx));
    }
    return Math.min(1.0f,  (progress + subprogress)/(float)(split.getLength()));
  }
{code}

> CombineFileRecordReader does not properly initialize child RecordReader
> -----------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1480
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1480
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Aaron Kimball
>            Assignee: Aaron Kimball
>         Attachments: MAPREDUCE-1480.2.patch, MAPREDUCE-1480.patch
>
>
> CombineFileRecordReader instantiates child RecordReader instances but never calls their
initialize() method to give them the proper TaskAttemptContext.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message