hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Shelukhin <ser...@hortonworks.com>
Subject Re: Review Request 62706: HIVE-17473 implement workload management pools
Date Wed, 18 Oct 2017 20:44:39 GMT


> On Oct. 11, 2017, 9:31 p.m., Zhiyuan Yang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java
> > Lines 250 (patched)
> > <https://reviews.apache.org/r/62706/diff/1/?file=1840588#file1840588line256>
> >
> >     Why add up parallelism of parent node with children node? Shouldn't parent's
paralleism be a sum of children's?

No, parallelism are tracked separately, see the doc


> On Oct. 11, 2017, 9:31 p.m., Zhiyuan Yang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java
> > Lines 275 (patched)
> > <https://reviews.apache.org/r/62706/diff/1/?file=1840588#file1840588line284>
> >
> >     This piece get really complicated right now. I think there is good chance this
can be prettier. Are you going you rewrite this (as you mentioned in jira)?

Yeah, it's going to be rewritten in the next patch


> On Oct. 11, 2017, 9:31 p.m., Zhiyuan Yang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java
> > Lines 322 (patched)
> > <https://reviews.apache.org/r/62706/diff/1/?file=1840588#file1840588line331>
> >
> >     unreachable statement?

it's reachable if internal versions don't match


> On Oct. 11, 2017, 9:31 p.m., Zhiyuan Yang wrote:
> > ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java
> > Lines 243 (patched)
> > <https://reviews.apache.org/r/62706/diff/1/?file=1840589#file1840589line246>
> >
> >     Why can user use non-leaf queue? The fact that sum of sub-queue can be less
than parent queue's resource looks weird. Is this by design?

See design doc :) Because percentages are percentages, and parallelisms are absolute (percentages
don't map well to whole AMs - hard to reason about how many AMs one will get and rounding
can produce weird behavior), percentages are from parent total, and parallelisms are independent.


- Sergey


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62706/#review187692
-----------------------------------------------------------


On Sept. 30, 2017, 12:57 a.m., Sergey Shelukhin wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62706/
> -----------------------------------------------------------
> 
> (Updated Sept. 30, 2017, 12:57 a.m.)
> 
> 
> Review request for hive, Zhiyuan Yang and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> see jira
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPool.java 4f2997b95b 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/UserPoolMapping.java PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 3f621271cc 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 7adf895077

>   service/src/java/org/apache/hive/service/server/HiveServer2.java 5cb973ca95 
> 
> 
> Diff: https://reviews.apache.org/r/62706/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message