Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id E9146200CBD for ; Wed, 21 Jun 2017 23:08:03 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id E80C2160BD0; Wed, 21 Jun 2017 21:08:03 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 35B97160BD5 for ; Wed, 21 Jun 2017 23:08:03 +0200 (CEST) Received: (qmail 19090 invoked by uid 500); 21 Jun 2017 21:08:02 -0000 Mailing-List: contact issues-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@spark.apache.org Received: (qmail 18994 invoked by uid 99); 21 Jun 2017 21:08:02 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Jun 2017 21:08:02 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id E59E1192712 for ; Wed, 21 Jun 2017 21:08:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id hXtyNEIENkjS for ; Wed, 21 Jun 2017 21:08:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 0AC555FC6C for ; Wed, 21 Jun 2017 21:08:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 7CEB4E08BB for ; Wed, 21 Jun 2017 21:08:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 398A421943 for ; Wed, 21 Jun 2017 21:08:00 +0000 (UTC) Date: Wed, 21 Jun 2017 21:08:00 +0000 (UTC) From: "Joseph K. Bradley (JIRA)" To: issues@spark.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (SPARK-21166) Automated ML persistence MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Wed, 21 Jun 2017 21:08:04 -0000 Joseph K. Bradley created SPARK-21166: ----------------------------------------- Summary: Automated ML persistence Key: SPARK-21166 URL: https://issues.apache.org/jira/browse/SPARK-21166 Project: Spark Issue Type: New Feature Components: ML Affects Versions: 2.2.0 Reporter: Joseph K. Bradley This JIRA is for discussing the possibility of automating ML persistence. Currently, custom save/load methods are written for every Model. However, we could design a mixin which provides automated persistence, inspecting model data and Params and reading/writing (known types) automatically. This was brought up in discussions with developers behind https://github.com/azure/mmlspark Some issues we will need to consider: * Providing generic mixin usable in most or all cases * Handling corner cases (strange Param types, etc.) * Backwards compatibility (loading models saved by old Spark versions) Because of backwards compatibility in particular, it may make sense to implement testing for that first, before we try to address automated persistence: [SPARK-15573] -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org For additional commands, e-mail: issues-help@spark.apache.org