hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4730) AM crashes due to OOM while serving up map task completion events
Date Mon, 22 Oct 2012 22:16:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481862#comment-13481862

Vinod Kumar Vavilapalli commented on MAPREDUCE-4730:

Patch looks good.

Can you try writing a simple test for EventFetcher? You can mock umbilicalProtocol, shuffleScheduler
and reporter I suppose. Then you can validate your current change also. Let me know if it
becomes too cumbersome.

bq. The only issue I ran into was a significant number of maps and reduces failed because
they timed out trying to establish a connection to the AM.
This is new. I don't remember us running into it when we ran AMScalability. Can you file a
bug, more details will be great to have.
> AM crashes due to OOM while serving up map task completion events
> -----------------------------------------------------------------
>                 Key: MAPREDUCE-4730
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4730
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: applicationmaster, mrv2
>    Affects Versions: 0.23.3
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Blocker
>         Attachments: MAPREDUCE-4730.patch, MAPREDUCE-4730.patch
> We're seeing a repeatable OOM crash in the AM for a task with around 30000 maps and 3000
reducers.  Details to follow.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message