drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Altekruse" <altekruseja...@gmail.com>
Subject Re: Review Request 30701: DRILL-2173 partition queries for dynamic partition pruning
Date Sat, 14 Mar 2015 00:10:00 GMT


> On Feb. 6, 2015, 9:21 p.m., Jason Altekruse wrote:
> > exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionExplorer.java,
line 35
> > <https://reviews.apache.org/r/30701/diff/2/?file=851938#file851938line35>
> >
> >     I am considering making it a requirement that implimentors of this interface
return a partition description that can be fed back into the method unmodified to continue
to traverse down the partition tree.
> 
> Jason Altekruse wrote:
>     I had originally declared this interface to take in and return Strings, but this
required boilerplate in the UDF to convert the input holders to strings. So I changed the
interface to take the VarCharHolders and started to change the return type to be a VarCharHolder
array. Unfortunately this would have forced me to expose off heap allocation to the storage
plugin, which seemed undesireable. Please let me know your thoughts.
>     
>     Changing the docs as proposed above would allow for more generic code with just boilerplate
to convert the returned strings from the first call to the interface back into varchar holders
in subsequent calls to traverse down the heirarchy. It would remove the need for storge plugin
specific decisions about whether or not the original partition could/must be concatenated
with the sub-partition and what characters UDF writers should glue them together with.

fixed


- Jason


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30701/#review71361
-----------------------------------------------------------


On Feb. 28, 2015, 12:18 a.m., Jason Altekruse wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30701/
> -----------------------------------------------------------
> 
> (Updated Feb. 28, 2015, 12:18 a.m.)
> 
> 
> Review request for drill, Jacques Nadeau, Mehant Baid, and Parth Chandra.
> 
> 
> Bugs: DRILL-2173
>     https://issues.apache.org/jira/browse/DRILL-2173
> 
> 
> Repository: drill-git
> 
> 
> Description
> -------
> 
> Adds a new interface for UDFs to access partition information. Together with 2060 which
allows constant expression folding this will allow UDFs that
> 
> 
> Diffs
> -----
> 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/DrillFuncHolder.java 279c428

>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionConverter.java 0127e6e

>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/impl/DirectoryExplorers.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/interpreter/InterpreterEvaluator.java
0fe36cb 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java e413921

>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java c881432 
>   exec/java-exec/src/main/java/org/apache/drill/exec/ops/UdfUtilities.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractStoragePlugin.java
b032fce 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionExplorer.java PRE-CREATION

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionNotFoundException.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePlugin.java ef5978c

>   exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginPartitionExplorer.java
PRE-CREATION 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/StoragePluginRegistry.java
5d0eed6 
>   exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemPlugin.java
7a1f61e 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/interp/ExpressionInterpreterTest.java
PRE-CREATION 
>   exec/java-exec/src/test/java/org/apache/drill/exec/fn/interp/TestConstantFolding.java
PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/30701/diff/
> 
> 
> Testing
> -------
> 
> Some unit tests on the functionality have been run, still a work in progress so no full
mvn build run yet
> 
> 
> Thanks,
> 
> Jason Altekruse
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message