[ https://issues.apache.org/jira/browse/MAPREDUCE-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13480289#comment-13480289
]
Jason Lowe commented on MAPREDUCE-4730:
---------------------------------------
Update on testing, I was able to test this (along with the fix for MAPREDUCE-4733) using a
sleep job with 20000 maps and 3000 reduces on a cluster big enough to mass-launch the map
and reduce phases. The AM with a 1.5GB slot size stayed up during the job, where previously
it failed even with a larger slot.
The only issue I ran into was a significant number of maps and reduces failed because they
timed out trying to establish a connection to the AM. I suspected the AM could have been
busy garbage collecting and causing the delays, so I bumped up the AM size to 3G and it ran
smoothly with no connection timeout failures from any tasks.
> AM crashes due to OOM while serving up map task completion events
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-4730
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4730
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mrv2
> Affects Versions: 0.23.3
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Priority: Blocker
> Attachments: MAPREDUCE-4730.patch, MAPREDUCE-4730.patch
>
>
> We're seeing a repeatable OOM crash in the AM for a task with around 30000 maps and 3000
reducers. Details to follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
|