Return-Path: X-Original-To: apmail-drill-commits-archive@www.apache.org Delivered-To: apmail-drill-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 8B56D1844E for ; Wed, 7 Oct 2015 17:06:47 +0000 (UTC) Received: (qmail 82857 invoked by uid 500); 7 Oct 2015 17:06:44 -0000 Delivered-To: apmail-drill-commits-archive@drill.apache.org Received: (qmail 82826 invoked by uid 500); 7 Oct 2015 17:06:44 -0000 Mailing-List: contact commits-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: commits@drill.apache.org Delivered-To: mailing list commits@drill.apache.org Received: (qmail 82817 invoked by uid 99); 7 Oct 2015 17:06:44 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 07 Oct 2015 17:06:44 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 42896E0329; Wed, 7 Oct 2015 17:06:44 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: amansinha@apache.org To: commits@drill.apache.org Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: drill git commit: DRILL-3901: Don't do early expansion of directory in the non-metadata-cache case because it already happens during ParquetGroupScan's metadata gathering operation. Date: Wed, 7 Oct 2015 17:06:44 +0000 (UTC) Repository: drill Updated Branches: refs/heads/master 64bf5c8e6 -> 6cd399de8 DRILL-3901: Don't do early expansion of directory in the non-metadata-cache case because it already happens during ParquetGroupScan's metadata gathering operation. Update unit test that was changed by a previous fix for DRILL-3788 Project: http://git-wip-us.apache.org/repos/asf/drill/repo Commit: http://git-wip-us.apache.org/repos/asf/drill/commit/6cd399de Tree: http://git-wip-us.apache.org/repos/asf/drill/tree/6cd399de Diff: http://git-wip-us.apache.org/repos/asf/drill/diff/6cd399de Branch: refs/heads/master Commit: 6cd399de82d18c1f8c2e4992b0a2c3f2d4fbbbcf Parents: 64bf5c8 Author: Aman Sinha Authored: Tue Oct 6 19:38:04 2015 -0700 Committer: Aman Sinha Committed: Wed Oct 7 09:43:54 2015 -0700 ---------------------------------------------------------------------- .../org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java | 4 +++- .../src/test/java/org/apache/drill/TestCTASPartitionFilter.java | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/drill/blob/6cd399de/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java ---------------------------------------------------------------------- diff --git a/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java b/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java index b494427..7f95024 100644 --- a/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java +++ b/exec/java-exec/src/main/java/org/apache/drill/exec/store/parquet/ParquetFormatPlugin.java @@ -223,7 +223,9 @@ public class ParquetFormatPlugin implements FormatPlugin{ } return new FileSelection(fileNames, true); } else { - return selection.minusDirectories(fs); + // don't expand yet; ParquetGroupScan's metadata gathering operation + // does that. + return selection; } } http://git-wip-us.apache.org/repos/asf/drill/blob/6cd399de/exec/java-exec/src/test/java/org/apache/drill/TestCTASPartitionFilter.java ---------------------------------------------------------------------- diff --git a/exec/java-exec/src/test/java/org/apache/drill/TestCTASPartitionFilter.java b/exec/java-exec/src/test/java/org/apache/drill/TestCTASPartitionFilter.java index 1f53a81..9886024 100644 --- a/exec/java-exec/src/test/java/org/apache/drill/TestCTASPartitionFilter.java +++ b/exec/java-exec/src/test/java/org/apache/drill/TestCTASPartitionFilter.java @@ -74,7 +74,7 @@ public class TestCTASPartitionFilter extends PlanTestBase { test("use dfs_test.tmp"); test(String.format("create table drill_3410 partition by (o_orderpriority) as select * from dfs_test.`%s/multilevel/parquet`", TEST_RES_PATH)); String query = "select * from drill_3410 where (o_orderpriority = '1-URGENT' and o_orderkey = 10) or (o_orderpriority = '2-HIGH' or o_orderkey = 11)"; - testIncludeFilter(query, 5, "Filter", 34); + testIncludeFilter(query, 1, "Filter", 34); } @Test