spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan Cutler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-23159) Update Cloudpickle to match version 0.4.2
Date Fri, 19 Jan 2018 18:58:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-23159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16332752#comment-16332752
] 

Bryan Cutler commented on SPARK-23159:
--------------------------------------

I can work on this

> Update Cloudpickle to match version 0.4.2
> -----------------------------------------
>
>                 Key: SPARK-23159
>                 URL: https://issues.apache.org/jira/browse/SPARK-23159
>             Project: Spark
>          Issue Type: Improvement
>          Components: PySpark
>    Affects Versions: 2.3.0
>            Reporter: Bryan Cutler
>            Priority: Major
>
> Update PySpark's version of Cloudpickle to match version 0.4.2.  The reasons for doing
this are:
>  * Pick up bug fixes, improvements with newer version
>  * Match a specific version as close as possible (Spark has additional changes that might
be necessary) to make future upgrades easier
> There are newer versions of Cloudpickle that can fix bugs with NamedTuple pickling (that
Spark currently has workarounds for), but these include other changes that need some verification
before bringing into Spark.  Upgrading first to 0.4.2 will help make this verification easier.
> Discussion on the mailing list: http://apache-spark-developers-list.1001551.n3.nabble.com/Thoughts-on-Cloudpickle-Update-td23188.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message