spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apache Spark (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (SPARK-24044) Explicitly print out skipped tests from unittest module
Date Sun, 22 Apr 2018 09:02:00 GMT

     [ https://issues.apache.org/jira/browse/SPARK-24044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Apache Spark reassigned SPARK-24044:
------------------------------------

    Assignee:     (was: Apache Spark)

> Explicitly print out skipped tests from unittest module
> -------------------------------------------------------
>
>                 Key: SPARK-24044
>                 URL: https://issues.apache.org/jira/browse/SPARK-24044
>             Project: Spark
>          Issue Type: Test
>          Components: PySpark
>    Affects Versions: 2.4.0
>            Reporter: Hyukjin Kwon
>            Priority: Major
>
> There was an actual issue, SPARK-23300, and we fixed this by manually checking if the
package is installed. This way needed duplicated codes and could only check dependencies.
There are many conditions, for example, Python version specific or other packages like NumPy.
 I think this is something we should fix.
> `unittest` module can print out the skipped messages but these were swallowed so far
in our own testing script. This PR prints out the messages below after sorted.
> It would be nicer if we remove the duplications and print out all the skipped tests.
For example, as below:
> This PR proposes to remove duplicated dependency checking logics and also print out skipped
tests from unittests. 
> For example, as below:
> {code}
> Skipped tests in pyspark.sql.tests with pypy:
>     test_createDataFrame_column_name_encoding (pyspark.sql.tests.ArrowTests) ... skipped
'Pandas >= 0.19.2 must be installed; however, it was not found.'
>     test_createDataFrame_does_not_modify_input (pyspark.sql.tests.ArrowTests) ... skipped
'Pandas >= 0.19.2 must be installed; however, it was not found.'
> ...
> Skipped tests in pyspark.sql.tests with python3:
>     test_createDataFrame_column_name_encoding (pyspark.sql.tests.ArrowTests) ... skipped
'PyArrow >= 0.8.0 must be installed; however, it was not found.'
>     test_createDataFrame_does_not_modify_input (pyspark.sql.tests.ArrowTests) ... skipped
'PyArrow >= 0.8.0 must be installed; however, it was not found.'
> ...
> {code}
> Actual format can be a bit varied per the discussion in the PR. Please check out the
PR for exact format.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message