hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2722) Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is used
Date Fri, 22 Jul 2011 11:52:57 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ravi Gummadi updated MAPREDUCE-2722:
------------------------------------

    Attachment: MR2722.patch

Attaching patch fixing the bug.
The new counter values with this patch for the Gridmix simulated job are(for the example case
of previous comment):

100MB as hdfsBytesRead and 200MB as MapInputBytes.

Please review and provide your comments.

> Gridmix simulated job's map's hdfsBytesRead counter is wrong when compressed input is
used
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2722
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2722
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: contrib/gridmix
>            Reporter: Ravi Gummadi
>            Assignee: Ravi Gummadi
>         Attachments: MR2722.patch
>
>
> When compressed input was used by original job's map task, then the simulated job's map
task's hdfsBytesRead counter is wrong if compression emulation is enabled. This issue is because
hdfsBytesRead of map task of original job is considered as uncompressed map input size by
Gridmix.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message