spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From felixcheung <...@git.apache.org>
Subject [GitHub] spark pull request #20473: [SPARK-23300][TESTS] Prints out if Pandas and PyA...
Date Fri, 02 Feb 2018 07:23:00 GMT
Github user felixcheung commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20473#discussion_r165572746
  
    --- Diff: python/run-tests.py ---
    @@ -151,6 +152,68 @@ def parse_opts():
         return opts
     
     
    +def _check_dependencies(python_exec, modules_to_test):
    +    if "COVERAGE_PROCESS_START" in os.environ:
    +        # Make sure if coverage is installed.
    +        try:
    +            subprocess_check_output(
    +                [python_exec, "-c", "import coverage"],
    +                stderr=open(os.devnull, 'w'))
    +        except:
    +            print_red("Coverage is not installed in Python executable '%s' "
    +                      "but 'COVERAGE_PROCESS_START' environment variable is set, "
    +                      "exiting." % python_exec)
    +            sys.exit(-1)
    +
    +    # If we should test 'pyspark-sql', it checks if PyArrow and Pandas are installed
and
    +    # explicitly prints out. See SPARK-23300.
    +    if pyspark_sql in modules_to_test:
    +        # Hyukjin: I think here is not the best place to leave versions for extra dependencies.
    --- End diff --
    
    that, I'd agree...


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message