hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2365) Add counters for FileInputFormat (BYTES_READ) and FileOutputFormat (BYTES_WRITTEN)
Date Mon, 07 Mar 2011 22:20:03 GMT
Add counters for FileInputFormat (BYTES_READ) and FileOutputFormat (BYTES_WRITTEN)
----------------------------------------------------------------------------------

                 Key: MAPREDUCE-2365
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2365
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Owen O'Malley


MAP_INPUT_BYTES and MAP_OUTPUT_BYTES will be computed using the difference between FileSystem
counters before and after each next(K,V) and collect/write op.

In case compression is being used, these counters will represent the compressed data sizes.
The uncompressed size will
not be available.

This is not a direct back-port of 5710. (Counters will be computed in MapTask instead of in
individual RecordReaders).

0.20.100 ->
   New API -> MAP_INPUT_BYTES will be computed using this method
   Old API -> MAP_INPUT_BYTES will remain unchanged.

0.23 ->
   New API -> MAP_INPUT_BYTES will be computed using this method
   Old API -> MAP_INPUT_BYTES likely to use this method


--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message