hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sharad Agarwal (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5710) Counter MAP_INPUT_BYTES missing from new mapreduce api.
Date Tue, 05 May 2009 06:03:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12705915#action_12705915
] 

Sharad Agarwal commented on HADOOP-5710:
----------------------------------------

I propose that we remove MAP_INPUT_BYTES from the framework counters as it is specific to
file based input formats only. With new api, the framework no more calculates this. It is
getting calculated in lib. So we can move  MAP_INPUT_BYTES to FileInputFormat class. That
seems to be its more logical place.

> Counter MAP_INPUT_BYTES missing from new mapreduce api.
> -------------------------------------------------------
>
>                 Key: HADOOP-5710
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5710
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>             Fix For: 0.21.0
>
>         Attachments: patch-5710.txt
>
>
> MapTask, with current new mapreduce api, does not maintain MAP_INPUT_BYTES counter. Since
RecordReader doesnot have getPos() api, it is not possible for the map task to maintain. Individual
record readers (LineRecordReader, SequenceFileRecordReader) could be modified to maintain
the same.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message