hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hong Tang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1317) Reducing memory consumption of rumen objects
Date Fri, 18 Dec 2009 11:34:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792441#action_12792441

Hong Tang commented on MAPREDUCE-1317:

Through YourKit profiling, we found two places where we could save memory:
- LoggedLocation - we should share references to the same LoggedLocation for the same preferred
location for different map tasks.
- LoggedTaskAttempt.hostName - we should keep a cache of all host names for the cluster and
share the references.

> Reducing memory consumption of rumen objects
> --------------------------------------------
>                 Key: MAPREDUCE-1317
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1317
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Hong Tang
> We have encountered OutOfMemoryErrors in mumak and gridmix when dealing with very large
jobs. The purpose of this jira is to optimze memory consumption of rumen produced job objects.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message