spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph K. Bradley (JIRA)" <>
Subject [jira] [Commented] (SPARK-19247) improve ml word2vec save/load
Date Tue, 17 Jan 2017 01:10:26 GMT


Joseph K. Bradley commented on SPARK-19247:

Is this an actual problem?  If this is just a nicety, I would prefer not to change it until
after we put it unit tests for backwards compatibility (SPARK-15573)

> improve ml word2vec save/load
> -----------------------------
>                 Key: SPARK-19247
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Asher Krim
> ml word2vec models can be somewhat large (~4gb is not uncommon). The current save implementation
saves the model as a single large datum, which can cause rpc issues and fail to save the model.
> On the loading side, there are issues with loading this large datum as well. This was
already solved for mllib word2vec in, but
the change was never ported to the ml word2vec implementation.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message