hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-812) Total number of splits/maps can be encoded as the first field while serializing splits
Date Tue, 28 Jul 2009 12:13:14 GMT
Total number of splits/maps can be encoded as the first field while serializing splits
--------------------------------------------------------------------------------------

                 Key: MAPREDUCE-812
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-812
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
          Components: jobtracker
            Reporter: Amar Kamat


To find out the total number of maps, the whole split file is deserialized and  then the checks
are made (num-maps = length of the split array). The issue is that if total number of splits
is more then unnecessarily load all the splits and then discard it. Instead we can encode
the total number of splits as the first field.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message