parquet-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tomwh...@apache.org
Subject git commit: PARQUET-25. Pushdown predicates only work with hardcoded arguments.
Date Wed, 23 Jul 2014 13:30:06 GMT
Repository: incubator-parquet-mr
Updated Branches:
  refs/heads/master f28423863 -> 4a07b3f54


PARQUET-25. Pushdown predicates only work with hardcoded arguments.

Pull request for Sandy Ryza's fix for PARQUET-25.

Author: Sandy Ryza <sandy.ryza@cloudera.com>

Closes #22 from tomwhite/PARQUET-25-unbound-record-filter-configurable and squashes the following
commits:

a9d3fdc [Sandy Ryza] PARQUET-25. Pushdown predicates only work with hardcoded arguments.


Project: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/commit/4a07b3f5
Tree: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/tree/4a07b3f5
Diff: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/diff/4a07b3f5

Branch: refs/heads/master
Commit: 4a07b3f545aaf60f0b1d6bba91ee22d214dfaff8
Parents: f284238
Author: Sandy Ryza <sandy.ryza@cloudera.com>
Authored: Wed Jul 23 14:29:35 2014 +0100
Committer: Tom White <tom@cloudera.com>
Committed: Wed Jul 23 14:29:35 2014 +0100

----------------------------------------------------------------------
 .../main/java/parquet/hadoop/ParquetInputFormat.java    | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/blob/4a07b3f5/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java
----------------------------------------------------------------------
diff --git a/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java b/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java
index b09c0ff..882d2f7 100644
--- a/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java
+++ b/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java
@@ -26,6 +26,7 @@ import java.util.List;
 import java.util.Map;
 import java.util.Set;
 
+import org.apache.hadoop.conf.Configurable;
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.fs.BlockLocation;
 import org.apache.hadoop.fs.FileStatus;
@@ -134,13 +135,18 @@ public class ParquetInputFormat<T> extends FileInputFormat<Void,
T> {
   public RecordReader<Void, T> createRecordReader(
       InputSplit inputSplit,
       TaskAttemptContext taskAttemptContext) throws IOException, InterruptedException {
-    ReadSupport<T> readSupport = getReadSupport(ContextUtil.getConfiguration(taskAttemptContext));
-    Class<?> unboundRecordFilterClass = getUnboundRecordFilter(ContextUtil.getConfiguration(taskAttemptContext));
+    Configuration conf = ContextUtil.getConfiguration(taskAttemptContext);
+    ReadSupport<T> readSupport = getReadSupport(conf);
+    Class<?> unboundRecordFilterClass = getUnboundRecordFilter(conf);
     if (unboundRecordFilterClass == null) {
       return new ParquetRecordReader<T>(readSupport);
     } else {
       try {
-        return new ParquetRecordReader<T>(readSupport, (UnboundRecordFilter)unboundRecordFilterClass.newInstance());
+        UnboundRecordFilter filter = (UnboundRecordFilter)unboundRecordFilterClass.newInstance();
+        if (filter instanceof Configurable) {
+          ((Configurable)filter).setConf(conf);
+        }
+        return new ParquetRecordReader<T>(readSupport, filter);
       } catch (InstantiationException e) {
         throw new BadConfigurationException("could not instantiate unbound record filter
class", e);
       } catch (IllegalAccessException e) {


Mime
View raw message