hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3667) Gridmix jobs are failing with OOM in reduce shuffle phase.
Date Wed, 01 Feb 2012 19:37:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198079#comment-13198079

Vinod Kumar Vavilapalli commented on MAPREDUCE-3667:

bq. Hadoop 0.20.204 reducers does not goes OOM, I have seen this in Hadoop.Next only.
That is right, Karam. We would ideally want YARN+MR to be able to run with the same configuration.
Unfortunately mrv2 reduce runtime has changed a lot, and it performs better in so many other
situations but worse in this particular case. The implementation has changed in such a way
that supporting 0.7 is not possible with the current set of jobs in gridmix without running
into OOM.

I agree that it is a regression but for now this should be fine. We'd ideally want to fix
this, will open a separate ticket, but the timelines for that fix are uncertain at this point.

> Gridmix jobs are failing with OOM in reduce shuffle phase.
> ----------------------------------------------------------
>                 Key: MAPREDUCE-3667
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3667
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Amol Kekre
>            Priority: Blocker
>             Fix For: 0.23.1
> Roll up bug for gridmix3 benchmark

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message