hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7494) Add muti node lookup support for better placement
Date Thu, 29 Mar 2018 03:58:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-7494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418411#comment-16418411
] 

Weiwei Yang commented on YARN-7494:
-----------------------------------

Hi [~sunilg]

I was proposing to have configuration like following:

{noformat}

<property>
  <name>yarn.scheduler.capacity.multi-node-sorting.polices</name>
  <value>resource-usage</value>
 </property>

<property>
  <name>yarn.scheduler.capacity.multi-node-sorting.policy.resource-usage.class</name>
  <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.placement.ResourceUsageSortingNodesPolicy</value>
</property>

<property>
  <name>yarn.scheduler.capacity.multi-node-sorting.policy.resource-usage.sorting-task.interval.ms</name>
  <value>3000</value>
</property>

{noformat}

Such fashion will be easy for user to plug their own policies. And this is how queues are
configured so should be easy for user to understand. What do you think? I am also open if
you can elaborate your config details (currently it cannot be cleanly seen from the patch),
I can vote yes as long as it addresses these problems. 

Thanks

 

> Add muti node lookup support for better placement
> -------------------------------------------------
>
>                 Key: YARN-7494
>                 URL: https://issues.apache.org/jira/browse/YARN-7494
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>            Priority: Major
>         Attachments: YARN-7494.001.patch, YARN-7494.002.patch, YARN-7494.003.patch, YARN-7494.004.patch,
YARN-7494.005.patch, YARN-7494.v0.patch, YARN-7494.v1.patch, multi-node-designProposal.png
>
>
> Instead of single node, for effectiveness we can consider a multi node lookup based on
partition to start with.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message