drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Altekruse" <altekruseja...@gmail.com>
Subject Re: Review Request 30701: DRILL-2173 partition queries for dynamic partition pruning
Date Fri, 03 Apr 2015 23:12:29 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30701/
-----------------------------------------------------------

(Updated April 3, 2015, 11:12 p.m.)


Review request for drill, Jacques Nadeau, Mehant Baid, Parth Chandra, and Venki Korukanti.


Changes
-------

Added UDFs for both case sensitivities as well as finding minimum directory (full list: maxdir,imaxdir,mindir,imindir),
Now generated with freemarker. Changed interface to take List instead of Collection to enforce
ordering (as these lists are expected to be used in parellel). Removed unused imports and
some general cleanup. SubDirectoryList was shared between the workspace and filesystem schemas,
so I pulled it out into a separate class.


Bugs: DRILL-2173
    https://issues.apache.org/jira/browse/DRILL-2173


Repository: drill-git


Description
-------

Adds a new interface for UDFs to access partition information. Together with 2060 which allows
constant expression folding this will allow UDFs that can query against partition information
and then scan a subset of data. Example use case, find the most recent directory and only
that partition worth of data.


Diffs (updated)
-----

  contrib/storage-hbase/src/main/java/org/apache/drill/exec/store/hbase/HBaseSchemaFactory.java
7b76092 
  contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/schema/HiveSchemaFactory.java
023517b 
  contrib/storage-mongo/src/main/java/org/apache/drill/exec/store/mongo/schema/MongoSchemaFactory.java
32c42ba 
  exec/java-exec/src/main/codegen/templates/DirectoryExplorers.java PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/FunctionConverter.java ab121b0

  exec/java-exec/src/main/java/org/apache/drill/exec/expr/fn/interpreter/InterpreterEvaluator.java
35c35ec 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/FragmentContext.java a4ac724 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/QueryContext.java 3b51a69 
  exec/java-exec/src/main/java/org/apache/drill/exec/ops/UdfUtilities.java f7a1a04 
  exec/java-exec/src/main/java/org/apache/drill/exec/store/AbstractSchema.java 90e3ef4 
  exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionExplorer.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionExplorerImpl.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/store/PartitionNotFoundException.java
PRE-CREATION 
  exec/java-exec/src/main/java/org/apache/drill/exec/store/SchemaPartitionExplorer.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/store/SubSchemaWrapper.java 2c0d8b8 
  exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/FileSystemSchemaFactory.java
4a3eba9 
  exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/SubDirectoryList.java PRE-CREATION

  exec/java-exec/src/main/java/org/apache/drill/exec/store/dfs/WorkspaceSchemaFactory.java
aeff09b 
  exec/java-exec/src/test/java/org/apache/drill/exec/fn/interp/TestConstantFolding.java PRE-CREATION

  exec/java-exec/src/test/java/org/apache/drill/exec/planner/TestDirectoryExplorerUDFs.java
PRE-CREATION 

Diff: https://reviews.apache.org/r/30701/diff/


Testing
-------

Test have been run on a very recent version, made a few minor cleanup edits since, waiting
on another run, but do not anticipate issues.


Thanks,

Jason Altekruse


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message