hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2411) [Capacity Scheduler] support simple user and group mappings to queues
Date Fri, 15 Aug 2014 07:35:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-2411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14098293#comment-14098293
] 

Hadoop QA commented on YARN-2411:
---------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12661960/YARN-2411.1.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 1 new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 core tests{color}.  The patch failed these unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:

                  org.apache.hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector
                  org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStoreZKClientConnections

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-YARN-Build/4630//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-YARN-Build/4630//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4630//console

This message is automatically generated.

> [Capacity Scheduler] support simple user and group mappings to queues
> ---------------------------------------------------------------------
>
>                 Key: YARN-2411
>                 URL: https://issues.apache.org/jira/browse/YARN-2411
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Ram Venkatesh
>            Assignee: Ram Venkatesh
>         Attachments: YARN-2411.1.patch
>
>
> YARN-2257 has a proposal to extend and share the queue placement rules for the fair scheduler
and the capacity scheduler. This is a good long term solution to streamline queue placement
of both schedulers but it has core infra work that has to happen first and might require changes
to current features in all schedulers along with corresponding configuration changes, if any.

> I would like to propose a change with a smaller scope in the capacity scheduler that
addresses the core use cases for implicitly mapping jobs that have the default queue or no
queue specified to specific queues based on the submitting user and user groups. It will be
useful in a number of real-world scenarios and can be migrated over to the unified scheme
when YARN-2257 becomes available.
> The proposal is to add two new configuration options:
> yarn.scheduler.capacity.queue-mappings-override.enable 
> A boolean that controls if user-specified queues can be overridden by the mapping, default
is false.
> and,
> yarn.scheduler.capacity.queue-mappings
> A string that specifies a list of mappings in the following format (default is "" which
is the same as no mapping)
> <map_specifier>:<source_attribute>:<queue_name>[,<map_specifier>:<source_attribute>:<queue_name>]*
> map_specifier := user (u) | group (g)
> source_attribute := user | group | %user
> queue_name := the name of the mapped queue | %user | %primary_group
> The mappings will be evaluated left to right, and the first valid mapping will be used.
If the mapped queue does not exist, or the current user does not have permissions to submit
jobs to the mapped queue, the submission will fail.
> Example usages:
> 1. user1 is mapped to queue1, group1 is mapped to queue2
> u:user1:queue1,g:group1:queue2
> 2. To map users to queues with the same name as the user:
> u:%user:%user
> I am happy to volunteer to take this up.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message