Return-Path: X-Original-To: apmail-parquet-commits-archive@minotaur.apache.org Delivered-To: apmail-parquet-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6D3421059D for ; Wed, 23 Jul 2014 13:30:28 +0000 (UTC) Received: (qmail 77616 invoked by uid 500); 23 Jul 2014 13:30:28 -0000 Delivered-To: apmail-parquet-commits-archive@parquet.apache.org Received: (qmail 77595 invoked by uid 500); 23 Jul 2014 13:30:28 -0000 Mailing-List: contact commits-help@parquet.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@parquet.incubator.apache.org Delivered-To: mailing list commits@parquet.incubator.apache.org Received: (qmail 77554 invoked by uid 99); 23 Jul 2014 13:30:28 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jul 2014 13:30:28 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 23 Jul 2014 13:30:26 +0000 Received: (qmail 75889 invoked by uid 99); 23 Jul 2014 13:30:06 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Jul 2014 13:30:06 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 3D5999B11D9; Wed, 23 Jul 2014 13:30:06 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: tomwhite@apache.org To: commits@parquet.incubator.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: git commit: PARQUET-25. Pushdown predicates only work with hardcoded arguments. Date: Wed, 23 Jul 2014 13:30:06 +0000 (UTC) X-Virus-Checked: Checked by ClamAV on apache.org Repository: incubator-parquet-mr Updated Branches: refs/heads/master f28423863 -> 4a07b3f54 PARQUET-25. Pushdown predicates only work with hardcoded arguments. Pull request for Sandy Ryza's fix for PARQUET-25. Author: Sandy Ryza Closes #22 from tomwhite/PARQUET-25-unbound-record-filter-configurable and squashes the following commits: a9d3fdc [Sandy Ryza] PARQUET-25. Pushdown predicates only work with hardcoded arguments. Project: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/commit/4a07b3f5 Tree: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/tree/4a07b3f5 Diff: http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/diff/4a07b3f5 Branch: refs/heads/master Commit: 4a07b3f545aaf60f0b1d6bba91ee22d214dfaff8 Parents: f284238 Author: Sandy Ryza Authored: Wed Jul 23 14:29:35 2014 +0100 Committer: Tom White Committed: Wed Jul 23 14:29:35 2014 +0100 ---------------------------------------------------------------------- .../main/java/parquet/hadoop/ParquetInputFormat.java | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-parquet-mr/blob/4a07b3f5/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java ---------------------------------------------------------------------- diff --git a/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java b/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java index b09c0ff..882d2f7 100644 --- a/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java +++ b/parquet-hadoop/src/main/java/parquet/hadoop/ParquetInputFormat.java @@ -26,6 +26,7 @@ import java.util.List; import java.util.Map; import java.util.Set; +import org.apache.hadoop.conf.Configurable; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.BlockLocation; import org.apache.hadoop.fs.FileStatus; @@ -134,13 +135,18 @@ public class ParquetInputFormat extends FileInputFormat { public RecordReader createRecordReader( InputSplit inputSplit, TaskAttemptContext taskAttemptContext) throws IOException, InterruptedException { - ReadSupport readSupport = getReadSupport(ContextUtil.getConfiguration(taskAttemptContext)); - Class unboundRecordFilterClass = getUnboundRecordFilter(ContextUtil.getConfiguration(taskAttemptContext)); + Configuration conf = ContextUtil.getConfiguration(taskAttemptContext); + ReadSupport readSupport = getReadSupport(conf); + Class unboundRecordFilterClass = getUnboundRecordFilter(conf); if (unboundRecordFilterClass == null) { return new ParquetRecordReader(readSupport); } else { try { - return new ParquetRecordReader(readSupport, (UnboundRecordFilter)unboundRecordFilterClass.newInstance()); + UnboundRecordFilter filter = (UnboundRecordFilter)unboundRecordFilterClass.newInstance(); + if (filter instanceof Configurable) { + ((Configurable)filter).setConf(conf); + } + return new ParquetRecordReader(readSupport, filter); } catch (InstantiationException e) { throw new BadConfigurationException("could not instantiate unbound record filter class", e); } catch (IllegalAccessException e) {