spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From travishegner <...@git.apache.org>
Subject [GitHub] spark pull request #19191: [SPARK-21958][ML] Word2VecModel save: transform d...
Date Mon, 11 Sep 2017 15:53:53 GMT
GitHub user travishegner opened a pull request:

    https://github.com/apache/spark/pull/19191

    [SPARK-21958][ML] Word2VecModel save: transform data in the cluster

    ## What changes were proposed in this pull request?
    
    Change a data transformation while saving a Word2VecModel to happen with distributed data
instead of local driver data.
    
    ## How was this patch tested?
    
    Unit tests for the ML sub-component still passes.
    Running this patch against v2.2.0 in a fully distributed production cluster allows a 4.0G
model to save and load correctly, where it would not do so without the patch.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/travishegner/spark master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/19191.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #19191
    
----
commit 5f4ce997f6f30cd0d59bc2e2f4396f495c3c0fd8
Author: Travis Hegner <thegner@trilliumit.com>
Date:   2017-09-08T14:51:53Z

    [SPARK-21958][ML] Word2VecModel save: transform data in the cluster

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


Mime
View raw message