spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiangrui Meng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-6288) Pyrolite calls hashCode to cache previously serialized objects
Date Fri, 01 May 2015 21:58:06 GMT

    [ https://issues.apache.org/jira/browse/SPARK-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14524052#comment-14524052
] 

Xiangrui Meng commented on SPARK-6288:
--------------------------------------

irmen is testing the path bintray -> jcenter -> maven central now. Hopefully this works.

> Pyrolite calls hashCode to cache previously serialized objects
> --------------------------------------------------------------
>
>                 Key: SPARK-6288
>                 URL: https://issues.apache.org/jira/browse/SPARK-6288
>             Project: Spark
>          Issue Type: Improvement
>          Components: PySpark
>    Affects Versions: 1.0.2, 1.1.1, 1.2.1, 1.3.0
>            Reporter: Xiangrui Meng
>            Assignee: Josh Rosen
>         Attachments: Screen Shot 2015-03-13 at 10.45.35 AM.png
>
>
> https://github.com/irmen/Pyrolite/blob/v2.0/java/src/net/razorvine/pickle/Pickler.java#L140
> This operation could be quite expensive, compared to serializing the object directly,
because hashCode usually needs to access all data stored in the object. Maybe we should disable
this feature by default.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message