hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantinos Karanasos (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7839) Check node capacity before placing in the Algorithm
Date Mon, 29 Jan 2018 19:20:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343850#comment-16343850

Konstantinos Karanasos commented on YARN-7839:

I think the change makes sense, [~asuresh].

However, what about the case that a node seems full but a container is about to finish (and
will be finished until the allocate is done)? Should we completely reject such nodes, or simply
give higher priority to nodes that already have available resources?
{quote}getPreferredNodeIterator(CandidateNodeSet<N> candidateNodeSet)
[~cheersyang], despite the naming, as far as I know, the candidateNodeSet is currently always
only a single node...

> Check node capacity before placing in the Algorithm
> ---------------------------------------------------
>                 Key: YARN-7839
>                 URL: https://issues.apache.org/jira/browse/YARN-7839
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Arun Suresh
>            Priority: Major
> Currently, the Algorithm assigns a node to a requests purely based on if the constraints
are met. It is later in the scheduling phase that the Queue capacity and Node capacity are
checked. If the request cannot be placed because of unavailable Queue/Node capacity, the request
is retried by the Algorithm.
> For clusters that are running at high utilization, we can reduce the retries if we perform
the Node capacity check in the Algorithm as well. The Queue capacity check and the other user
limit checks can still be handled by the scheduler (since queues and other limits are tied
to the scheduler, and not scheduler agnostic)

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message