hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3490) RMContainerAllocator counts failed maps towards Reduce ramp up
Date Tue, 03 Jan 2012 18:52:41 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13178910#comment-13178910
] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-3490:
----------------------------------------------------

bq. I am attaching a patch which drastically simplify this, without the need to add new events.
Also I have removed the completedMaps and completedReduces counts in RMContainerAllocator.
Arun/Vinod - see if this make sense ?
Thanks for the explanation, Sharad. Makes sense. Obviously we missed the big picture here
:) At any rate, this code definitely needs some cleanup, way too complicated for my simple
mind to track all of it ;)

Thanks for the update, Arun. Looking at the patch now.
                
> RMContainerAllocator counts failed maps towards Reduce ramp up
> --------------------------------------------------------------
>
>                 Key: MAPREDUCE-3490
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3490
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mr-am, mrv2
>    Affects Versions: 0.23.0
>            Reporter: Siddharth Seth
>            Assignee: Sharad Agarwal
>            Priority: Blocker
>         Attachments: MAPREDUCE-3490.patch, MAPREDUCE-3490.patch, MAPREDUCE-3490.patch,
MAPREDUCE-3490.patch, MAPREDUCE-3490.patch, MAPREDUCE-3490.patch, MR-3490-alternate.patch,
MR-3490-alternate1.patch
>
>
> The RMContainerAllocator does not differentiate between failed and successful maps while
calculating whether reduce tasks are ready to launch. Failed tasks are also counted towards
total completed tasks. 
> Example. 4 failed maps, 10 total maps. Map%complete = 4/14 * 100 instead of being 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message