spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph K. Bradley (JIRA)" <>
Subject [jira] [Created] (SPARK-21166) Automated ML persistence
Date Wed, 21 Jun 2017 21:08:00 GMT
Joseph K. Bradley created SPARK-21166:

             Summary: Automated ML persistence
                 Key: SPARK-21166
             Project: Spark
          Issue Type: New Feature
          Components: ML
    Affects Versions: 2.2.0
            Reporter: Joseph K. Bradley

This JIRA is for discussing the possibility of automating ML persistence.  Currently, custom
save/load methods are written for every Model.  However, we could design a mixin which provides
automated persistence, inspecting model data and Params and reading/writing (known types)
automatically.  This was brought up in discussions with developers behind

Some issues we will need to consider:
* Providing generic mixin usable in most or all cases
* Handling corner cases (strange Param types, etc.)
* Backwards compatibility (loading models saved by old Spark versions)

Because of backwards compatibility in particular, it may make sense to implement testing for
that first, before we try to address automated persistence: [SPARK-15573]

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message