hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Resolved: (HADOOP-875) memory model is not accurate enough for map side sorts
Date Thu, 16 Aug 2007 04:53:30 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Devaraj Das resolved HADOOP-875.
--------------------------------

    Resolution: Fixed

Improved this as part of HADOOP-1698.

> memory model is not accurate enough for map side sorts
> ------------------------------------------------------
>
>                 Key: HADOOP-875
>                 URL: https://issues.apache.org/jira/browse/HADOOP-875
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.10.0
>            Reporter: Owen O'Malley
>            Assignee: Devaraj Das
>
> I configured a sort (IdentityMapper) with large compressed values with the map output
buffer (io.sort.mb) set to 300mb and the child jvm heap size set to 900mb and some of the
maps were running out of memory deterministically. I suspect that some part of the data path
is not handling large values well and is consuming large amounts of ram.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message