hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <>
Subject [jira] [Commented] (HIVE-7739) TestSparkCliDriver should not use includeQueryFiles
Date Fri, 15 Aug 2014 04:58:18 GMT


Brock Noland commented on HIVE-7739:

Yeah I see what you are saying. My thought was that until we have a bunch of, perhaps the
majority, of the tests enabled, the most common case will be running tests which we are not
sure if they should be added to spark.query.files. Finding tests which currently pass is then
the following process:

* add the tests to spark.query.files
* run the tests with -Dqfile
* remove the ones that don't pass because they do something we don't yet support

as opposed to just running the tests with -Dqfile.

My thought on how to handle your use case is:

from itests dir:

mvn test ... -Dqfile="$(perl -pe 's@\\ *\n@@g' src/test/resources/
| awk -F= '/spark.query.files=/ {print $2}')"

I am open to both options. Thoughts?

> TestSparkCliDriver should not use includeQueryFiles
> ---------------------------------------------------
>                 Key: HIVE-7739
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Brock Noland
>            Assignee: Brock Noland
>         Attachments: HIVE-7739.1-spark.patch
> By using includesQueryFile TestSparkCliDriver cannot be used by -Dqfile or -Dqfile_regex.
These options are very useful so let's remove it.
> spark.query.files in will still be used when run via the
pre-commit tests to generate -Dqfiles

This message was sent by Atlassian JIRA

View raw message