Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3D44D19507 for ; Tue, 29 Mar 2016 18:22:31 +0000 (UTC) Received: (qmail 11535 invoked by uid 500); 29 Mar 2016 18:22:31 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 11436 invoked by uid 500); 29 Mar 2016 18:22:31 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 11427 invoked by uid 99); 29 Mar 2016 18:22:30 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Mar 2016 18:22:30 +0000 Received: from mail-yw0-f172.google.com (mail-yw0-f172.google.com [209.85.161.172]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 813571A0622 for ; Tue, 29 Mar 2016 18:22:30 +0000 (UTC) Received: by mail-yw0-f172.google.com with SMTP id h65so29707491ywe.0 for ; Tue, 29 Mar 2016 11:22:30 -0700 (PDT) X-Gm-Message-State: AD7BkJLd+syWKtu4J97HZEljHMbXdlm22W/hCALL28J3tBgt4lK6rGcCdFWoEI3LeveQdFnyuqJ47BXndmITMg== MIME-Version: 1.0 X-Received: by 10.129.94.194 with SMTP id s185mr2046276ywb.93.1459275749769; Tue, 29 Mar 2016 11:22:29 -0700 (PDT) Received: by 10.13.192.2 with HTTP; Tue, 29 Mar 2016 11:22:29 -0700 (PDT) In-Reply-To: References: Date: Tue, 29 Mar 2016 14:22:29 -0400 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Flink ML 1.0.0 - Saving and Loading Models to Score a Single Feature Vector From: Suneel Marthi To: user@flink.apache.org Content-Type: multipart/alternative; boundary=001a1147086eb1fa3b052f341cb0 --001a1147086eb1fa3b052f341cb0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable U may want to use FlinkMLTools.persist() methods which use TypeSerializerFormat and don't enforce IOReadableWritable. On Tue, Mar 29, 2016 at 2:12 PM, Sourigna Phetsarath < gna.phetsarath@teamaol.com> wrote: > Till, > > Thank you for your reply. > > Having this issue though, WeightVector does not extend IOReadWriteable: > > *public* *class* SerializedOutputFormat<*T* *extends* IOReadableWritable> > > *case* *class* WeightVector(weights: Vector, intercept: Double) *extends* > Serializable {} > > > However, I will use the approach to write out the weights as text. > > > On Tue, Mar 29, 2016 at 5:01 AM, Till Rohrmann > wrote: > >> Hi Gna, >> >> there are no utilities yet to do that but you can do it manually. In the >> end, a model is simply a Flink DataSet which you can serialize to some >> file. Upon reading this DataSet you simply have to give it to your >> algorithm to be used as the model. The following code snippet illustrate= s >> this approach: >> >> mlr.fit(inputDS, parameters) >> >> // write model to disk using the SerializedOutputFormat >> mlr.weightsOption.get.write(new SerializedOutputFormat[WeightVector], "p= ath") >> >> // read the serialized model from disk >> val model =3D env.readFile(new SerializedInputFormat[WeightVector], "pat= h") >> >> // set the read model for the MLR algorithm >> mlr.weightsOption =3D model >> >> Cheers, >> Till >> =E2=80=8B >> >> On Tue, Mar 29, 2016 at 10:46 AM, Simone Robutti < >> simone.robutti@radicalbit.io> wrote: >> >>> To my knowledge there is nothing like that. PMML is not supported in an= y >>> form and there's no custom saving format yet. If you really need a quic= k >>> and dirty solution, it's not that hard to serialize the model into a fi= le. >>> >>> 2016-03-28 17:59 GMT+02:00 Sourigna Phetsarath < >>> gna.phetsarath@teamaol.com>: >>> >>>> Flinksters, >>>> >>>> Is there an example of saving a Trained Model, loading a Trained Model >>>> and then scoring one or more feature vectors using Flink ML? >>>> >>>> All of the examples I've seen have shown only sequential fit and >>>> predict. >>>> >>>> Thank you. >>>> >>>> -Gna >>>> -- >>>> >>>> >>>> *Gna Phetsarath*System Architect // AOL Platforms // Data Services // >>>> Applied Research Chapter >>>> 770 Broadway, 5th Floor, New York, NY 10003 >>>> o: 212.402.4871 // m: 917.373.7363 >>>> vvmr: 8890237 aim: sphetsarath20 t: @sourigna >>>> >>>> * * >>>> >>> >>> >> > > > -- > > > *Gna Phetsarath*System Architect // AOL Platforms // Data Services // > Applied Research Chapter > 770 Broadway, 5th Floor, New York, NY 10003 > o: 212.402.4871 // m: 917.373.7363 > vvmr: 8890237 aim: sphetsarath20 t: @sourigna > > * * > --001a1147086eb1fa3b052f341cb0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
U may want to use FlinkMLTools.persist() methods which use= TypeSerializerFormat and don't enforce IOReadableWritable.


=

On Tue, Mar= 29, 2016 at 2:12 PM, Sourigna Phetsarath <gna.phetsarath@teamaol= .com> wrote:
Till,

Thank you for your reply. =C2=A0

=
Having this issue though, WeightVector does not extend IOReadWri= teable:

publi= c class Serialized= OutputFormat<T ex= tends IOReadableWritable>=C2=A0

case class WeightVector(weights: Vector, int= ercept: Double) extends Serializable {}


However, I will use the approach to write out the weig= hts as text.


On Tue, Mar 29, 2016 a= t 5:01 AM, Till Rohrmann <trohrmann@apache.org> wrote:

Hi Gna,

there are no utilities yet to d= o that but you can do it manually. In the end, a model is simply a Flink DataSet which you can serialize to some file. Upon re= ading this DataSet you simply have to give it to yo= ur algorithm to be used as the model. The following code snippet illustrate= s this approach:

mlr.fit(inputDS, par=
ameters)

// write model to disk using the SerializedOutputFormat
mlr.weightsOption.get.write(new SerializedOutputFormat[WeightVector], "=
;path")

// read the serialized model from disk
val model =3D env.readFile(new SerializedInputFormat[WeightVector], "p=
ath")

// set the read model for the MLR algorithm
mlr.weightsOption =3D model

Cheers,
Till

=E2=80=8B

On Tue, Mar 29, 2016 at 10:46 AM, Si= mone Robutti <simone.robutti@radicalbit.io> wrote= :
To my knowledge there = is nothing like that. PMML is not supported in any form and there's no = custom saving format yet. If you really need a quick and dirty solution, it= 's not that hard to serialize the model into a file.=C2=A0

2016-03-28 17:= 59 GMT+02:00 Sourigna Phetsarath <gna.phetsarath@teamaol.com&= gt;:
Flinksters,<= div>
Is there an example of saving a Trained Model, loading a= Trained Model and then scoring one or more feature vectors using Flink ML?=

All of the examples I've seen have shown only= sequential fit and predict.

Thank you.
=
-Gna
--

Gna Phetsarath
<= font size=3D"2">System Architect // AOL P= latforms // Data Services // Applied Research Chapter
770 Broadway, 5th = Floor, New York, NY 10003
o: 212.402.4871 // m: 917.373.7363
vvmr:=C2=A088902= 37=C2=A0aim: sphetsarath20 t: @sourigna<= /p>







--
=

Gna Phetsarath
<= font size=3D"2">System Architect // AOL P= latforms // Data Services // Applied Research Chapter
770 Broadway, 5th = Floor, New York, NY 10003
o: 212.402.4871 // m: 917.373.7363
vvmr:=C2=A088902= 37=C2=A0aim: sphetsarath20 t: @sourigna<= /p>



--001a1147086eb1fa3b052f341cb0--