mahout-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Palumbo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAHOUT-1786) Make classes implements Serializable for Spark 1.5+
Date Mon, 19 Dec 2016 18:43:58 GMT

     [ https://issues.apache.org/jira/browse/MAHOUT-1786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Palumbo updated MAHOUT-1786:
-----------------------------------
    Sprint:   (was: Jan/Feb-2016)

> Make classes implements Serializable for Spark 1.5+
> ---------------------------------------------------
>
>                 Key: MAHOUT-1786
>                 URL: https://issues.apache.org/jira/browse/MAHOUT-1786
>             Project: Mahout
>          Issue Type: Improvement
>          Components: Math
>    Affects Versions: 0.11.0
>            Reporter: Michel Lemay
>            Priority: Blocker
>              Labels: performance
>             Fix For: 0.13.0
>
>
> Spark 1.5 comes with a new very efficient serializer that uses code generation.  It is
twice as fast as kryo.  When using mahout, we have to set KryoSerializer because some classes
aren't serializable otherwise.  
> I suggest to declare Math classes as "implements Serializable" where needed.  For instance,
to use coocurence package in spark 1.5, we had to modify AbstractMatrix, AbstractVector, DenseVector
and SparseRowMatrix to make it work without Kryo.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message