drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau" <jacques.dr...@gmail.com>
Subject Re: Review Request 31107: Ability to make PartitionSender multithreaded - useful in case of LocalExchange being enabled, as it allows to deal with high volume of incoming data
Date Sun, 15 Mar 2015 20:28:47 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31107/#review76510
-----------------------------------------------------------

Ship it!


Few little comments.  Otherwise, looks good to ship.


exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java
<https://reviews.apache.org/r/31107/#comment124062>

    shouldn't this parameter be instanceCount?  instanceNumber seems like an index.



exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorStats.java
<https://reviews.apache.org/r/31107/#comment124063>

    missing description of what isClean means.



exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java
<https://reviews.apache.org/r/31107/#comment124064>

    returning null here seems weird.  When would that happen?


- Jacques Nadeau


On March 9, 2015, 4:31 p.m., Yuliya Feldman wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31107/
> -----------------------------------------------------------
> 
> (Updated March 9, 2015, 4:31 p.m.)
> 
> 
> Review request for drill, Chris Westin, Jacques Nadeau, Steven Phillips, and Venki Korukanti.
> 
> 
> Bugs: DRILL-2210
>     https://issues.apache.org/jira/browse/DRILL-2210
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> In addition to description
> 
> Fixed few classes that did not handle multithreading well
> Added/Changed some Stats behavior to allow stats merge from multiple threads, since again
this class is not suitable to be used in multithreaded environment
> Introduced new decorator class to handle multi thrteading (or not)  to minimize changes
to ParitionSenderRootExec class
> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java 7cc350e

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java e413921

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorStats.java 0e9da0e 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/IteratorValidator.java
64cf7c5 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SendingAccountor.java
7af7b65 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
a23bd7a 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/Partitioner.java
5ed9c39 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java
71ffd41 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Materializer.java
961b603 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java
bbfbbcb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java 0fb10ff

>   exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java
3d3e96f 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java 99c6ab8 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java
4aaaa78 
>   exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/partitionsender/TestPartitionSender.java
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/31107/diff/
> 
> 
> Testing
> -------
> 
> Still need to provide Unit Tests.
> 
> Functional tests are passing
> 
> Performance tests were run and look promising for some queries
> 
> 
> Thanks,
> 
> Yuliya Feldman
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message