hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8424) Support fair scheduler user queue mapping in non-impersonation mode
Date Sun, 12 Oct 2014 07:29:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168555#comment-14168555
] 

Szehon Ho commented on HIVE-8424:
---------------------------------

The code still looks good but I just checked some of the non-code changes, and had few logistic
suggestions, sorry for not seeing this earlier.  Instead of changing pom.xml, can we add TestScheduler
to itests/hadoop-2?  It will make it cleaner to consolidate this logics instead of having
multiple ways to exclude a test from hadoop-1.

Also while we are at it, the new conf description could be more understandable, something
like:

"This setting determines how YARN fair scheduler is configured when impersonation is off.
 If enabled, then logged-in user will be used to determine the fair scheduler queue for submitted
jobs.  Otherwise, user 'hive' will be used."

Please correct me if my understanding is wrong.  + [~leftylev] for any input on new conf property
as well. 


> Support fair scheduler user queue mapping in non-impersonation mode
> -------------------------------------------------------------------
>
>                 Key: HIVE-8424
>                 URL: https://issues.apache.org/jira/browse/HIVE-8424
>             Project: Hive
>          Issue Type: Improvement
>          Components: Shims
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>         Attachments: HIVE-8424.1.patch, HIVE-8424.patch
>
>
> Under non-impersonation mode, all MR jobs run as the hive system user. The default scheduler
queue mapping is one queue per user. This is problematic for users who use the queues to regulate
and track their MR resource usage.
> Yarn exposes an API to retrieve the fair scheduler queue mapping, which we can use to
set the appropriate MR queue for the current user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message