drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Altekruse" <altekruseja...@gmail.com>
Subject Re: Review Request 30701: DRILL-2173 partition queries for dynamic partition pruning
Date Sat, 07 Feb 2015 00:21:22 GMT


> On Feb. 6, 2015, 9:21 p.m., Jason Altekruse wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionExplorer.java,
line 35
> > <https://reviews.apache.org/r/30701/diff/2/?file=851938#file851938line35>
> >
> >     I am considering making it a requirement that implimentors of this interface
return a partition description that can be fed back into the method unmodified to continue
to traverse down the partition tree.

I had originally declared this interface to take in and return Strings, but this required
boilerplate in the UDF to convert the input holders to strings. So I changed the interface
to take the VarCharHolders and started to change the return type to be a VarCharHolder array.
Unfortunately this would have forced me to expose off heap allocation to the storage plugin,
which seemed undesireable. Please let me know your thoughts.

Changing the docs as proposed above would allow for more generic code with just boilerplate
to convert the returned strings from the first call to the interface back into varchar holders
in subsequent calls to traverse down the heirarchy. It would remove the need for storge plugin
specific decisions about whether or not the original partition could/must be concatenated
with the sub-partition and what characters UDF writers should glue them together with.


- Jason


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30701/#review71361
-----------------------------------------------------------


On Feb. 6, 2015, 2:05 a.m., Jason Altekruse wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30701/
> -----------------------------------------------------------
> 
> (Updated Feb. 6, 2015, 2:05 a.m.)
> 
> 
> Review request for drill, Jacques Nadeau, Mehant Baid, and Parth Chandra.
> 
> 
> Bugs: DRILL-2173
>     https://issues.apache.org/jira/browse/DRILL-2173
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> Adds a new interface for UDFs to access partition information. Together with 2060 which
allows constant expression folding this will allow UDFs that
> 
> 
> Diffs
> -----
> 
>   exec/interpreter/src/test/java/org/apache/drill/exec/expr/ExpressionInterpreterTest.java
a94ef94 
>   exec/interpreter/src/test/java/org/apache/drill/exec/expr/TestConstantFolding.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillFuncHolder.java 279c428

>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionConverter.java 0127e6e

>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/DirectoryExplorers.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/interpreter/InterpreterEvaluator.java
0fe36cb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java e413921

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java c881432 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/UdfUtilities.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillRuleSets.java
3b7adca 
>   exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/OptiqForkedConstantReduxRule.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractStoragePlugin.java
b032fce 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionExplorer.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionNotFoundException.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePlugin.java ef5978c

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginPartitionExplorer.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java
5d0eed6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
db6c0c7 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/WorkManager.java 99c6ab8 
>   exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java 378e81a

> 
> Diff: https://reviews.apache.org/r/30701/diff/
> 
> 
> Testing
> -------
> 
> Some unit tests on the functionality have been run, still a work in progress so no full
mvn build run yet
> 
> 
> Thanks,
> 
> Jason Altekruse
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message