hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5710) Counter MAP_INPUT_BYTES missing from new mapreduce api.
Date Thu, 21 May 2009 05:58:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12711476#action_12711476
] 

Sharad Agarwal commented on HADOOP-5710:
----------------------------------------

Patch looks good. Few minor points:
should we rename the group and counter to more readable and intuitive value. Say Group name
as "FileInputFormatCounters" and counter name as "BYTES_READ", Because anyway we are calculating
the bytes read from the file, not really the bytes input to the map (there may be some bytes
skipped while reading so these two may not be same).
include a deprecated warning in the old Counters API while mapping the old name to new name.

> Counter MAP_INPUT_BYTES missing from new mapreduce api.
> -------------------------------------------------------
>
>                 Key: HADOOP-5710
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5710
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.21.0
>
>         Attachments: patch-5710-1.txt, patch-5710.txt
>
>
> MapTask, with current new mapreduce api, does not maintain MAP_INPUT_BYTES counter. Since
RecordReader doesnot have getPos() api, it is not possible for the map task to maintain. Individual
record readers (LineRecordReader, SequenceFileRecordReader) could be modified to maintain
the same.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message