hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3931) Bug in MapTask.MapOutputBuffer.collect leads to an unnecessary and harmful 'reset'
Date Mon, 11 Aug 2008 20:54:46 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Owen O'Malley updated HADOOP-3931:
----------------------------------

      Resolution: Fixed
    Hadoop Flags: [Reviewed]
          Status: Resolved  (was: Patch Available)

I just committed this. Thanks, Chris!

> Bug in MapTask.MapOutputBuffer.collect leads to an unnecessary and harmful 'reset'
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-3931
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3931
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.17.1
>            Reporter: Arun C Murthy
>            Assignee: Chris Douglas
>            Priority: Blocker
>             Fix For: 0.17.2
>
>         Attachments: HADOOP-3931_0_20080809.patch
>
>
> The unnecessary reset of the io.sort.mb buffer when bufindex==bufvoid leads to wrong,
very large, temporary key buffer being created and passed along to the Buffer.write. This
causes MapBufferTooSmallException as the Buffer thinks there isn't enough space to write this
record into. If a combiner is defined the record does not go through the combiner and hits
the disk directly.
> It's already fixed in hadoop-0.18.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message