hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-3931) Bug in MapTask.MapOutputBuffer.collect leads to an unnecessary and harmful 'reset'
Date Sat, 09 Aug 2008 19:22:45 GMT
Bug in MapTask.MapOutputBuffer.collect leads to an unnecessary and harmful 'reset'
----------------------------------------------------------------------------------

                 Key: HADOOP-3931
                 URL: https://issues.apache.org/jira/browse/HADOOP-3931
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
    Affects Versions: 0.17.1
            Reporter: Arun C Murthy
            Assignee: Chris Douglas
            Priority: Blocker
             Fix For: 0.17.2


The unnecessary reset of the io.sort.mb buffer when bufindex==bufvoid leads to wrong, very
large, temporary key buffer being created and passed along to the Buffer.write. This causes
MapBufferTooSmallException as the Buffer thinks there isn't enough space to write this record
into. If a combiner is defined the record does not go through the combiner and hits the disk
directly.

It's already fixed in hadoop-0.18.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message