hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlo Curino (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1990) Track time-to-allocation for different size containers
Date Tue, 26 Aug 2014 23:15:59 GMT

    [ https://issues.apache.org/jira/browse/YARN-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14111535#comment-14111535

Carlo Curino commented on YARN-1990:

We had a simple implementation for this, using QueueMetrics and maintaining a map of delays
(using SampleQuantile), tracking start-end 
of the wait-time by catching reserve() and unreserve() calls.  In our test environment, it
didn't seem to matter much. 

We might further investigate, and post patches after YARN-1051 is committed.

> Track time-to-allocation for different size containers 
> -------------------------------------------------------
>                 Key: YARN-1990
>                 URL: https://issues.apache.org/jira/browse/YARN-1990
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
> Allocation of Large Containers are notoriously problematic, as smaller containers can
more easily grab resources. 
> The proposal for this JIRA is to maintain a map of container sizes, and time-to-allocation,
that can be used as:
> * general insight on cluster behavior, 
> * to inform the reservation-system, and allows us to "account for delays" in allocation,
so that the user reservation is respected regardless the size of containers requested.

This message was sent by Atlassian JIRA

View raw message