hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-8424) Support fair scheduler user queue mapping in non-impersonation mode
Date Sun, 12 Oct 2014 22:28:33 GMT

    [ https://issues.apache.org/jira/browse/HIVE-8424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168823#comment-14168823
] 

Szehon Ho commented on HIVE-8424:
---------------------------------

As per my limited understanding, YARN fair-scheduler maps user to queue for resource-management.
 The flag will determine which user Hive will use to determine the YARN queue when impersonation
is off.  As you said, its 'hive' if flag if false, logged-in user if flag is true.  Otherwise,
its always logged-in user.

Behind the scenes, the queue lookup method is called by Hive, but is YARN code.  Hope that
captures it.

> Support fair scheduler user queue mapping in non-impersonation mode
> -------------------------------------------------------------------
>
>                 Key: HIVE-8424
>                 URL: https://issues.apache.org/jira/browse/HIVE-8424
>             Project: Hive
>          Issue Type: Improvement
>          Components: Shims
>            Reporter: Mohit Sabharwal
>            Assignee: Mohit Sabharwal
>         Attachments: HIVE-8424.1.patch, HIVE-8424.patch
>
>
> Under non-impersonation mode, all MR jobs run as the hive system user. The default scheduler
queue mapping is one queue per user. This is problematic for users who use the queues to regulate
and track their MR resource usage.
> Yarn exposes an API to retrieve the fair scheduler queue mapping, which we can use to
set the appropriate MR queue for the current user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message