hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas Nair" <the...@hortonworks.com>
Subject Re: Review Request 34776: HIVE-4239 : Remove lock on compilation stage
Date Thu, 04 Jun 2015 22:31:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34776/#review86727
-----------------------------------------------------------



ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java
<https://reviews.apache.org/r/34776/#comment138788>

    shoudl this also be static ?



ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java
<https://reviews.apache.org/r/34776/#comment138789>

    also make static ?



service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java
<https://reviews.apache.org/r/34776/#comment138791>

    HiveSessionProxy is the one that does the doAs().
    
    I looked some more, it looks like this should not cause a problem with the safeguards
added in HIVE-7890. (but I think we should treat that change as a safeguard, as I am not sure
of the performance implications of that additional check every time).
    
    However, with this approach, we would end up creating a new Hive object in most cases,
if there are large number of users. Also the checks done in "Hive.get(conf);" to see if new
object needs to be created is not trivial.
    
    The cases were sessions are used in parallel is rare, so paying this performance penalty
for the more common case is not really justified IMO.
    
    I think we should just use locks around metastore client class for that. I think thats
something we should pursue in a different jira.



ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezProcContext.java
<https://reviews.apache.org/r/34776/#comment138792>

    Thanks for cleaning up the utils and threadlocal use!



ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java
<https://reviews.apache.org/r/34776/#comment138790>

    can't everything in the GenTezUtils be made static and then we don't need to create any
instances of it ?
    Maybe make the constructor of GenTezUtils private.


- Thejas Nair


On June 2, 2015, 7:07 p.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34776/
> -----------------------------------------------------------
> 
> (Updated June 2, 2015, 7:07 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d733d71 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 5dac29f 
>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/RemoveDynamicPruningBySize.java 5d01311

>   ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezProcContext.java adc31ae 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java 0edfc5d 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezWork.java 6db8220 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TezCompiler.java 56707af 
>   ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 37b6d6f 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 343c68e 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java a29e5d1

>   service/src/test/org/apache/hive/service/cli/CLIServiceTest.java b4d517f 
> 
> Diff: https://reviews.apache.org/r/34776/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message