hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlo Curino (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
Date Tue, 26 Aug 2014 23:15:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111535#comment-14111535
] 

Carlo Curino commented on YARN-1990:
------------------------------------

We had a simple implementation for this, using QueueMetrics and maintaining a map of delays
(using SampleQuantile), tracking start-end 
of the wait-time by catching reserve() and unreserve() calls.  In our test environment, it
didn't seem to matter much. 

We might further investigate, and post patches after YARN-1051 is committed.

> Track time-to-allocation for different size containers 
> -------------------------------------------------------
>
>                 Key: YARN-1990
>                 URL: https://issues.apache.org/jira/browse/YARN-1990
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>
> Allocation of Large Containers are notoriously problematic, as smaller containers can
more easily grab resources. 
> The proposal for this JIRA is to maintain a map of container sizes, and time-to-allocation,
that can be used as:
> * general insight on cluster behavior, 
> * to inform the reservation-system, and allows us to "account for delays" in allocation,
so that the user reservation is respected regardless the size of containers requested.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message