spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kyle Kelley (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-6883) Fork pyspark's cloudpickle as a separate dependency
Date Mon, 13 Apr 2015 17:22:12 GMT
Kyle Kelley created SPARK-6883:
----------------------------------

             Summary: Fork pyspark's cloudpickle as a separate dependency
                 Key: SPARK-6883
                 URL: https://issues.apache.org/jira/browse/SPARK-6883
             Project: Spark
          Issue Type: Improvement
          Components: PySpark
            Reporter: Kyle Kelley


IPython, pyspark, picloud/multyvac/cloudpipe all rely on cloudpickle from various sources
(cloud, pyspark, and multyvac correspondingly). It would be great to have this as a separately
maintained project that can:

* Work with Python3
* Add tests!
* Use higher order pickling (when on Python3)
* Be installed with pip

We're starting this off at the PyCon sprints under https://github.com/cloudpipe/cloudpickle.
We'd like to coordinate with PySpark to make it work across all the above mentioned projects.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message