hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tao Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
Date Fri, 08 Dec 2017 11:42:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283419#comment-16283419

Tao Yang commented on YARN-7494:

Thanks for the patch. [~sunilg]
Some thoughts from my side:
* Agree with 2) from [~leftnoteasy]. Sorting requirement may be different in some scenarios.
For example, opportunistic containers would prefer considering node utilization to unallocated
resource. I think we should support expandable sorting library and AppPlacementAllocator can
choose from it.
* This patch iterates all partition nodes to create new PartitionBasedCandidateNodeSet for
every schedule process in CapacityScheduler#getCandidateNodeSet. I think we can keep a single
instance to avoid always creating the same set.
* This patch remain as it is to iterates all nodes and trigger the schedule process for every
node in CapacityScheduler#schedule. IIUC Is it better to move multiNodePlacementEnabled condition
branch from CapacityScheduler#getCandidateNodeSet to CapacityScheduler#schedule and iterates
all partitions to trigger the schedule process when multi-node-placement enabled ?

> Add muti node lookup support for better placement
> -------------------------------------------------
>                 Key: YARN-7494
>                 URL: https://issues.apache.org/jira/browse/YARN-7494
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-7494.v0.patch
> Instead of single node, for effectiveness we can consider a multi node lookup based on
partition to start with.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message