Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8CE65200BE4 for ; Wed, 21 Dec 2016 21:38:17 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 8B797160B26; Wed, 21 Dec 2016 20:38:17 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id AB940160B18 for ; Wed, 21 Dec 2016 21:38:16 +0100 (CET) Received: (qmail 66350 invoked by uid 500); 21 Dec 2016 20:38:15 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 66336 invoked by uid 99); 21 Dec 2016 20:38:15 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 21 Dec 2016 20:38:15 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 41372CDA91 for ; Wed, 21 Dec 2016 20:38:15 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.38 X-Spam-Level: ** X-Spam-Status: No, score=2.38 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, MIME_QP_LONG_LINE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id P-7F-Z6FOuGs for ; Wed, 21 Dec 2016 20:38:14 +0000 (UTC) Received: from mail-qk0-f195.google.com (mail-qk0-f195.google.com [209.85.220.195]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 85BA05F474 for ; Wed, 21 Dec 2016 20:38:13 +0000 (UTC) Received: by mail-qk0-f195.google.com with SMTP id t184so10427676qkd.1 for ; Wed, 21 Dec 2016 12:38:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:references:from:in-reply-to:message-id:date:to :content-transfer-encoding:mime-version; bh=AAZDAUMNfhXK/0cRXyZqwXuT8o4DMJfirkq0b9FwPTo=; b=usudAfAd65/Lhmb/uJoi6WPuGc061UprjXqkuVqUYCP4c49NZt266pphq7NxkgCUWj +bEQ3ygc/Vj5bCOUMaTbxLxw3StJY0jESmbY/Ii8BIUiK1zAkR6IqwXYTpAs9N88OGZb STUrxIBBoRDlZkDEJBeG7om+3mJMm48JMBSawbsqJewaq0eprZi8sAnpT8Zn7RuPVc6z RLt1W1TGjm9f4CKgP9rufM1tyceDxPS54l1GCnQPZAdoJrid5PNPnmEdSPjrDlWMt6kz q0B3CZfsDwl/APbi7WzXHOtX5q3wp0wXrZ9hugj6B+RJzy2KDVclFydsYNjaUrxnFU3l Aljw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:references:from:in-reply-to:message-id :date:to:content-transfer-encoding:mime-version; bh=AAZDAUMNfhXK/0cRXyZqwXuT8o4DMJfirkq0b9FwPTo=; b=Qvz/mB+QEFx5Yl/jjSBALmaTrRMOw/Akd/sjGYmqooi4kPMkOj6M3CKMBB4L3kyheX c1dNyjf6TJGV898RBSWD9YTKSIa0ZT2s7DzI+TmQ467VEfsa3bmbvrrBP81KQtTJis3M Ya9i4pQxvKblfLsxjB8KTxQ/yYFNBjr02W0GAQdYxonLXiJzRv0kTfoNTsse1aLaTslU /CuMyYj27UiujlExaMCC5ElTT/vSwL4aoXjSDTI514DvIk+0x0igvosajuVFnLmlpGYe D4CbG3veMqFXIFNl7+nEcdVcoELJCIrR9oYC68cP0pzV1Tbjn3SqYD9UqQGByPGOE6Rn s6fw== X-Gm-Message-State: AIkVDXLwr7RO6jui1znxPR7fWmXkEhOBYSCYmkDFnHgGJPDjgYgFBwr3h3a5UqwwZSJEQg== X-Received: by 10.55.164.5 with SMTP id n5mr6751569qke.307.1482352692593; Wed, 21 Dec 2016 12:38:12 -0800 (PST) Received: from [192.168.1.42] (r190-134-127-93.dialup.adsl.anteldata.net.uy. [190.134.127.93]) by smtp.gmail.com with ESMTPSA id h47sm16372970qtc.27.2016.12.21.12.38.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 21 Dec 2016 12:38:11 -0800 (PST) Subject: Re: FlinkML and DataStream API References: From: dromitlabs@gmail.com Content-Type: multipart/alternative; boundary=Apple-Mail-BA53BA5E-10B9-406F-8272-FC95BABE8228 X-Mailer: iPhone Mail (14B100) In-Reply-To: Message-Id: <05E78703-E7D2-48AD-9634-2E56A86BB5F8@gmail.com> Date: Wed, 21 Dec 2016 17:38:09 -0300 To: user@flink.apache.org Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) archived-at: Wed, 21 Dec 2016 20:38:17 -0000 --Apple-Mail-BA53BA5E-10B9-406F-8272-FC95BABE8228 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I'm interested in that code you mentioned too, I hope you can find it. Regards, Matt > On Dec 21, 2016, at 17:12, Theodore Vasiloudis wrote: >=20 > Hello M=C3=A4ki,=20 >=20 > I think what you would like to do is train a model using batch, and use th= e Flink streaming API as a way to serve your model and make predictions.=20 >=20 > While we don't have an integrated way to do that in FlinkML currently, I d= efinitely think that's possible. I know Marton Balassi has been working on s= omething like this for the ALS algorithm, but I can't find the code right no= w on mobile. =20 > The general idea is to keep your model as state and use it to make predict= ions on a stream of incoming data.=20 >=20 > Model serving is definitely something we'll be working on in the future, I= 'll have a master student working on exactly that next semester.=20 >=20 > --=20 > Sent from a mobile device. May contain autocorrect errors. >=20 >> On Dec 21, 2016 5:24 PM, "M=C3=A4ki Hanna" wrote= : >> Hi, >>=20 >> =20 >>=20 >> I=E2=80=99m wondering if there is a way to use FlinkML and make predictio= ns continuously for test data coming from a DataStream. >>=20 >> =20 >>=20 >> I know FlinkML only supports the DataSet API (batch) at the moment, but i= s there a way to convert a DataStream into DataSets? I=E2=80=99m thinking of= something like >>=20 >> =20 >>=20 >> (0. fit model in batch mode) >>=20 >> 1. window the DataStream >>=20 >> 2. convert the windowed stream to DataSets >>=20 >> 3. use the FlinkML methods to make predictions >>=20 >> =20 >>=20 >> BR, >>=20 >> Hanna >>=20 >> =20 >>=20 >> Disclaimer: This message and any attachments thereto are intended solely f= or the addressed recipient(s) and may contain confidential information. If y= ou are not the intended recipient, please notify the sender by reply e-mail a= nd delete the e-mail (including any attachments thereto) without producing, d= istributing or retaining any copies thereof. Any review, dissemination or ot= her use of, or taking of any action in reliance upon, this information by pe= rsons or entities other than the intended recipient(s) is prohibited. Thank y= ou. --Apple-Mail-BA53BA5E-10B9-406F-8272-FC95BABE8228 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
I'm interested in that code= you mentioned too, I hope you can find it.

Regards= ,
Matt

On Dec 21, 2016, at 17:12, Theodore Vasiloudi= s <theodoros.vasiloudis= @gmail.com> wrote:

Hello M=C3=A4ki, 

I think what you would like to do is train a model using batch, and use t= he Flink streaming API as a way to serve your model and make predictions.&nb= sp;

While we don't have a= n integrated way to do that in FlinkML currently, I definitely think that's p= ossible. I know Marton Balassi has been working on something like this for t= he ALS algorithm, but I can't find the code right now on mobile.  
The general idea is to keep your model as state and use i= t to make predictions on a stream of incoming data. 

Model serving is definitely something we'll= be working on in the future, I'll have a master student working on exactly t= hat next semester. 

--
Sent from a mobile device. May contain autocorrect errors.

On D= ec 21, 2016 5:24 PM, "M=C3=A4ki Hanna" <Hanna.Maki@comptel.com> wrote:

Hi,

 

I=E2=80=99m wondering if there i= s a way to use FlinkML and make predictions continuously for test data comin= g from a DataStream.

 

I know FlinkML only supports the= DataSet API (batch) at the moment, but is there a way to convert a DataStre= am into DataSets? I=E2=80=99m thinking of something like

 

(0. fit model in batch mode)<= /u>

1. window the DataStream<= u>

2. convert the windowed stream t= o DataSets

3. use the FlinkML methods to ma= ke predictions

 

BR,

Hanna

 

Disclaimer: This message and any attachments thereto are intended solely for= the addressed recipient(s) and may contain confidential information. If you= are not the intended recipient, please notify the sender by reply e-mail an= d delete the e-mail (including any attachments thereto) without producing, distributing or retaining any c= opies thereof. Any review, dissemination or other use of, or taking of any a= ction in reliance upon, this information by persons or entities other than t= he intended recipient(s) is prohibited. Thank you.
= --Apple-Mail-BA53BA5E-10B9-406F-8272-FC95BABE8228--