drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yuliya Feldman" <yufeldma...@gmail.com>
Subject Re: Review Request 31107: Ability to make PartitionSender multithreaded - useful in case of LocalExchange being enabled, as it allows to deal with high volume of incoming data
Date Tue, 17 Mar 2015 04:17:16 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31107/
-----------------------------------------------------------

(Updated March 16, 2015, 9:17 p.m.)


Review request for drill, Chris Westin, Jacques Nadeau, Steven Phillips, and Venki Korukanti.


Changes
-------

Latest (hopefully)

Addressing review comments
changed default settings for threads
added cost to metrics
added additional param to completely overwrite # of threads


Bugs: DRILL-2210
    https://issues.apache.org/jira/browse/DRILL-2210


Repository: drill-git


Description
-------

In addition to description

Fixed few classes that did not handle multithreading well
Added/Changed some Stats behavior to allow stats merge from multiple threads, since again
this class is not suitable to be used in multithreaded environment
Introduced new decorator class to handle multi thrteading (or not)  to minimize changes to
ParitionSenderRootExec class


Diffs (updated)
-----

  exec/java-exec/src/main/java/org/apache/drill/exec/compile/CodeCompiler.java 7cc350e 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java 108f5bb 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/OperatorStats.java 0e9da0e 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/config/IteratorValidator.java
64cf7c5 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/SendingAccountor.java 7af7b65

  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionSenderRootExec.java
ccbd289 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/Partitioner.java
5ed9c39 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerDecorator.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/partitionsender/PartitionerTemplate.java
1d9088a 
  exec/java-exec/src/main/java/org/apache/drill/exec/planner/fragment/Materializer.java 9b0944e

  exec/java-exec/src/main/java/org/apache/drill/exec/planner/physical/PlannerSettings.java
abbc910 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/DrillbitContext.java 0fb10ff 
  exec/java-exec/src/main/java/org/apache/drill/exec/server/options/SystemOptionManager.java
d821af8 
  exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java 99c6ab8 
  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/TestOptiqPlans.java 4aaaa78

  exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/partitionsender/TestPartitionSender.java
PRE-CREATION 

Diff: https://reviews.apache.org/r/31107/diff/


Testing
-------

Still need to provide Unit Tests.

Functional tests are passing

Performance tests were run and look promising for some queries


Thanks,

Yuliya Feldman


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message