Mailing-List: contact reviews-help@spark.apache.org; run by ezmlm
Precedence: bulk
From: HyukjinKwon <git@git.apache.org>
To: reviews@spark.apache.org
Reply-To: reviews@spark.apache.org
Message-ID: <git-pr-9659-spark@git.apache.org>
Subject: [GitHub] spark pull request: [SPARK-11676][SQL] Parquet filter tests
 all pa...
Content-Type: text/plain
Date: Thu, 12 Nov 2015 09:50:31 +0000 (UTC)

GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/9659

    [SPARK-11676][SQL] Parquet filter tests all pass if filters are not really pushed down

    Currently Parquet predicate tests all pass even if filters are not pushed down or this is disabled.
    
    In this PR, For checking evaluating filters, Simply it makes the expression from `expression.Filter` and then try to create filters just like Spark does.
    
    For checking the results, this manually accesses to the child rdd (of `expression.Filter`) and produces the results which should be filtered properly, and then compares it to expected values.
    
    Now, if filters are not pushed down or this is disabled, this throws exceptions.
    
    cc @liancheng 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark SPARK-11676

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/9659.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #9659
    
----
commit 1c6a5480df53d16fe38d14146deedaf035a6024c
Author: hyukjinkwon <gurwls223@gmail.com>
Date:   2015-11-12T09:31:41Z

    [SPARK-11676][SQL] Parquet filter tests all pass if filters are not really pushed down

commit 6b722ef4ad3a29a1bd0cde0961f6bc12638e2f31
Author: hyukjinkwon <gurwls223@gmail.com>
Date:   2015-11-12T09:37:27Z

    [SPARK-11676][SQL] Update indentation and comments

commit d007c3f996f1fd4e093a590741b92a4bc4072d41
Author: hyukjinkwon <gurwls223@gmail.com>
Date:   2015-11-12T09:39:05Z

    [SPARK-11676][SQL] Remove unused imports

commit 8cc842fa3f97c39e61fa072a916f7056a85be1b7
Author: hyukjinkwon <gurwls223@gmail.com>
Date:   2015-11-12T09:49:11Z

    [SPARK-111676][SQL] Set true to pushing down filters

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org