hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2876) ContainerAllocationExpirer appears to use the incorrect configs
Date Fri, 26 Aug 2011 15:22:29 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13091820#comment-13091820

Robert Joseph Evans commented on MAPREDUCE-2876:

The issue is with granularity and how far off from the expire interval the expire will actually
be.  In the worst case something will expire just under 1 monitor interval after it should
have.  The only issue would be if someone really needed that expiry to be much closer to the
actual expire interval then 1/3 of it.  I don't really see that as much of an issue for anything
we have right now.  

I would propose that we modify LivnessMonitor to store expiry in a priority queue instead
of a map.  Then it can sleep just the amount of time needed before the next one expires. 
It is fairly simple and I have done this type of thing in the past.  Then there is very little
error from the expire interval, and we don't have to worry about a monitor interval at all.

> ContainerAllocationExpirer appears to use the incorrect configs
> ---------------------------------------------------------------
>                 Key: MAPREDUCE-2876
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2876
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Robert Joseph Evans
>            Assignee: Robert Joseph Evans
>             Fix For: 0.23.0
> ContainerAllocationExpirer sets the expiration interval to be RMConfig.CONTAINER_LIVELINESS_MONITORING_INTERVAL
but uses AMLIVELINESS_MONITORING_INTERVAL as the interval.  This is very different from what
AMLivelinessMonitor does.
> There should be two configs RMConfig.CONTAINER_LIVELINESS_MONITORING_INTERVAL for the
monitoring interval and RMConfig.CONTAINER_EXPIRY_INTERVAL for the expiry.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message