spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From HyukjinKwon <>
Subject [GitHub] spark pull request #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpar...
Date Thu, 15 Mar 2018 04:24:42 GMT
GitHub user HyukjinKwon opened a pull request:

    [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests where possible

    ## What changes were proposed in this pull request? added
an useful util
    def sql_conf(self, pairs):
    to allow configuration set/unset within a block:
    with self.sql_conf({"spark.blah.blah.blah", "blah"})
        # test codes
    This PR proposes to use this util where possible in PySpark tests.
    Note that there look already few places affecting tests without restoring the original
value back in unittest classes.
    ## How was this patch tested?
    Manually tested via:
    ./run-tests --modules=pyspark-sql --python-executables=python2
    ./run-tests --modules=pyspark-sql --python-executables=python3

You can merge this pull request into a Git repository by running:

    $ git pull cleanup-sql-conf

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #20830
commit 89cf69be7ae00f571c51de402067928b663c5a45
Author: hyukjinkwon <gurwls223@...>
Date:   2018-03-15T04:16:18Z

    Use sql_conf util in PySpark tests where possible



To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message