Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Tue, 8 Jul 2014 10:50:04 +0000 (UTC)
From: "Patrick Liu (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12725693.1404720325176.218.1404816604783@arcas>
In-Reply-To: <JIRA.12725693.1404720325176@arcas>
References: <JIRA.12725693.1404720325176@arcas>
Subject: [jira] [Commented] (YARN-2257) Add user to queue mappings to
 automatically place users' apps into specific queues
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054818#comment-14054818 ] 

Patrick Liu commented on YARN-2257:
-----------------------------------

Hi, [~vinodkv].
Although I am new, I want challenges. And I will follow this thread. If I am capable of handling it, I will try.
I am using hadoop2.0.0-cdh4.6.0. If I want to fix this issue, which version should I download and try fix?

If we want to add the user->queue mapping ability in fair-scheduler, I think we should enhance the QueueManagerInfo, and add a map containing user->queue mapping in QueueManager.
The mapping will only affect the process when a new application submitted, so the 'handle' method in FairScheduler will be modified.
We can add a new method when 'APP_ADDED' occurred, and dispatch the app to the given queue defined in user->queue map.

However as [~kkambatl] suggested, we should transfer QueuePlacementRule feature to yarn. 
If so, how many existing features will be affected?

> Add user to queue mappings to automatically place users' apps into specific queues
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-2257
>                 URL: https://issues.apache.org/jira/browse/YARN-2257
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: scheduler
>            Reporter: Patrick Liu
>            Assignee: Vinod Kumar Vavilapalli
>              Labels: features
>
> Currently, the fair-scheduler supports two modes, default queue or individual queue for each user.
> Apparently, the default queue is not a good option, because the resources cannot be managed for each user or group.
> However, individual queue for each user is not good enough. Especially when connecting yarn with hive. There will be increasing hive users in a corporate environment. If we create a queue for a user, the resource management will be hard to maintain.
> I think the problem can be solved like this:
> 1. Define user->queue mapping in Fair-Scheduler.xml. Inside each queue, use aclSubmitApps to control user's ability.
> 2. Each time a user submit an app to yarn, if the user has mapped to a queue, the app will be scheduled to that queue; otherwise, the app will be submitted to default queue.
> 3. If the user cannot pass aclSubmitApps limits, the app will not be accepted.


--
This message was sent by Atlassian JIRA
(v6.2#6252)