hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-2774) Add counters to show number of key/values that have been sorted and merged in the maps and reduces
Date Sat, 02 Feb 2008 08:13:08 GMT
Add counters to show number of key/values that have been sorted and merged in the maps and
reduces
--------------------------------------------------------------------------------------------------

                 Key: HADOOP-2774
                 URL: https://issues.apache.org/jira/browse/HADOOP-2774
             Project: Hadoop Core
          Issue Type: Bug
            Reporter: Owen O'Malley


For each *pass* of the sort and merge, I would like a count of the number of records. So for
example, if the map output 100 records and they were sorted once, the counter would be 100.
If it spilled twice and was merged together, it would be 200. Clearly in a multi-level merge,
it may not be a multiple of the number of map output records. This would let the users easily
see if they have values like io.sort.mb or io.sort.factor set too low.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message