hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasad Mujumdar" <pras...@cloudera.com>
Subject Re: Review Request 26515: HIVE-8424 : Support fair scheduler user queue mapping in non-impersonation mode
Date Fri, 10 Oct 2014 01:24:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/26515/#review56090
-----------------------------------------------------------


Mostly look fine. Just one comment below.


service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java
<https://reviews.apache.org/r/26515/#comment96443>

    If we are calling this by default then the non-23 shims shouldn't thrown an exception.
Note that the config property  is true by default, that means every HS2 session throw a scary
warning message with Hadoop 1.x
    
    IMO the other shim implementations can be no-op.


- Prasad Mujumdar


On Oct. 9, 2014, 11:35 p.m., Mohit Sabharwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/26515/
> -----------------------------------------------------------
> 
> (Updated Oct. 9, 2014, 11:35 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-8424
>     https://issues.apache.org/jira/browse/HIVE-8424
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-8424 : Support fair scheduler user queue mapping in non-impersonation mode
> 
> Under non-impersonation mode, all MR jobs run as the hive system user. 
> The default scheduler queue mapping is one queue per user. This is problematic
> for users who use the queues to regulate and track their MR resource usage.
> 
> This patch uses the Yarn API to set the fair scheduler queue mapping for
> the current user, if impersonation is turned off.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 6a1a5f0542ca0723e9e415863ca74c8b43bcc96c

>   itests/hive-unit/pom.xml 077631a7d809e6d5fd79815fca8882898ee172e9 
>   itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestSchedulerQueue.java PRE-CREATION

>   pom.xml f5a25e5c86b0801d382abd66597be8d7ac69e426 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java f021870cf48c55a3d2dea4c825f1f25d0ab008f3

>   shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 52f02c1a7915125fedd7b756e1db4611cf81d966

>   shims/0.20S/src/main/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java 07946df1b0fc9064925f431fe1f0e2c020b41dfc

>   shims/0.23/pom.xml ad71fccc2f94f4523cbb3572fecc06e54528d630 
>   shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java 9829c1b54ed2c1cb3068a569ff47d897f6284933

>   shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java d56795f7be1cffa13ea7b3ea2cd50db90730c020

> 
> Diff: https://reviews.apache.org/r/26515/diff/
> 
> 
> Testing
> -------
> 
> Added unit test
> 
> 
> Thanks,
> 
> Mohit Sabharwal
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message