hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hung (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6686) Support for adding and removing queue mappings
Date Fri, 30 Jun 2017 02:30:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-6686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16069373#comment-16069373
] 

Jonathan Hung commented on YARN-6686:
-------------------------------------

Here's an initial patch which supports adding and removing from a global configuration csv
(e.g. yarn.scheduler.capacity.queue-mappings).

It's just an initial patch, there's still stuff missing, for example:
# adding the same functionality for queue configurations
# unit tests
but I wanted to post this early to see if there were any serious objections with it.

Basically in addition to {noformat}
<global-updates>
  <params>...</params>
</global-updates>{noformat} to specify updates to global configurations, you can also
do:
{noformat}
<global-updates>
  <addParams>...</addParams>
  <removeParams>...</removeParams>
</global-updates>{noformat} to specify configs you want to add/remove from the csv collection
for configuration keys. For example,
{noformat}
<global-updates>
  <addParams>
    <entry>
      <key>yarn.scheduler.capacity.queue-mappings</key>
      <value>u:user1:default</value>
    </entry>
  </addParams>
</global-updates>{noformat}
then 
{noformat}
<global-updates>
  <removeParams>
    <entry>
      <key>yarn.scheduler.capacity.queue-mappings</key>
      <value>u:user2:default</value>
    </entry>
  </removeParams>
</global-updates>{noformat} would add the user1->default queue mapping, then remove
the user2->default mapping. So if you started with {{u:user2:default,u:user3:default}}
as your mappings, you would end up with {{u:user3:default,u:user1:default}} in the end.

Any comments are welcome!

> Support for adding and removing queue mappings
> ----------------------------------------------
>
>                 Key: YARN-6686
>                 URL: https://issues.apache.org/jira/browse/YARN-6686
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Jonathan Hung
>            Assignee: Jonathan Hung
>         Attachments: YARN-6686-YARN-5734.001.patch
>
>
> Right now capacity scheduler uses UserGroupMappingPlacementRule to determine queue mappings.
This rule stores mappings in {{yarn.scheduler.capacity.queue-mappings}}. For users with a
large number of mappings, adding or removing queue mappings becomes infeasible.
> Need to come up with a way to add/remove individual mappings, for any/all different configured
placement rules.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message