Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id CD6A0200BDC for ; Wed, 14 Dec 2016 14:18:40 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id CA58D160B19; Wed, 14 Dec 2016 13:18:40 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EA0CE160B13 for ; Wed, 14 Dec 2016 14:18:39 +0100 (CET) Received: (qmail 94121 invoked by uid 500); 14 Dec 2016 13:18:39 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 94097 invoked by uid 99); 14 Dec 2016 13:18:39 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 14 Dec 2016 13:18:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 87202D09B7 for ; Wed, 14 Dec 2016 13:18:38 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.98 X-Spam-Level: * X-Spam-Status: No, score=1.98 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=data-artisans-com.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id a0sWDXpsqD1M for ; Wed, 14 Dec 2016 13:18:36 +0000 (UTC) Received: from mail-wj0-f174.google.com (mail-wj0-f174.google.com [209.85.210.174]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 320A45FCB2 for ; Wed, 14 Dec 2016 13:18:36 +0000 (UTC) Received: by mail-wj0-f174.google.com with SMTP id tk12so33997672wjb.3 for ; Wed, 14 Dec 2016 05:18:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=data-artisans-com.20150623.gappssmtp.com; s=20150623; h=from:mime-version:subject:date:references:to:in-reply-to:message-id; bh=G5W4BlaJlfg+c9QAP0hjbn5xMJuG/gzGEbfWNVeB5n8=; b=kDZ5rMkAu8dFN9PXcXhrr9uhhmqtJHP2Sfx1AZ2YX/qzYVLLEo0fN0Oio7MHnqNuxa Vllf4spD1TM42prmM+GPzC09rosWk4LvRNKG+crRNDLBDXvnNrlE/HMiyMpbejRpY2Px 9B8ozL3cceMUdbLiuffgAI/r2cGIeiYNx2U0rz13pn5MhiuHWVlutt2CA9M/S42zfhqO tkvEwzXaYf4+QFDWW3GZBflQKp5lt6Zgn9Qnm8MHGoEA9YpDnYAMcPXj8GixVAudIF6z +ZOQmKaT+J6G5CbDGrtapCeGnYU2gBivoA8rbdD9p/yOiauv1BzBPvNWNWAWzblyLXlg GCzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:mime-version:subject:date:references:to :in-reply-to:message-id; bh=G5W4BlaJlfg+c9QAP0hjbn5xMJuG/gzGEbfWNVeB5n8=; b=Zp7xp3K4LfYOKxVagddepR9I4Y/YdvtDcjSoBETM8seBHagDKMHPV8y2o5R88H8bx1 eO3nYOmiEqVvfBdIaNOKcT+KckoGPcR2JYi9szLv1tAvAJnVfJbS7O0jlkkCKT4p8Kvy v0nksz5TgJeD5J7Glc+DGH5gUCUh0WkZdG53LxkWxOzY5BFA7v/snZyoBeXE1B06EBvr 4RI+En/5ACHiw9jZdFhnf9sfwgsZUgVVvly68UbhjVuZpNGZsGQJYpRLfIv/frJ8mYs1 VlkLzEI0hICwHaLYyhnN5j1sQrWARsBSN9560nRdetnyJ+IFSR0fk6YC3rr6apSe5J9i Og4Q== X-Gm-Message-State: AKaTC01XkMrTjTr4BCK50ZPAca9R8n0MT3y0cbi6nB8brQiucrKtArsGIvrOsktfSBC7G2pm X-Received: by 10.28.149.135 with SMTP id x129mr7779475wmd.71.1481721514984; Wed, 14 Dec 2016 05:18:34 -0800 (PST) Received: from ?IPv6:2a02:8109:8080:1808:75f2:6dc4:41e9:3dda? ([2a02:8109:8080:1808:75f2:6dc4:41e9:3dda]) by smtp.gmail.com with ESMTPSA id c187sm7589772wmd.13.2016.12.14.05.18.34 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 14 Dec 2016 05:18:34 -0800 (PST) From: Stefan Richter Content-Type: multipart/alternative; boundary="Apple-Mail=_8D94F912-B2C0-4356-BB86-195AC0E62D63" Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) Subject: Re: Checkpointing Date: Wed, 14 Dec 2016 14:18:33 +0100 References: To: user@flink.apache.org In-Reply-To: Message-Id: <0B3DF43B-F0FE-4B4B-8B0B-44D9588F7FF4@data-artisans.com> X-Mailer: Apple Mail (2.3251) archived-at: Wed, 14 Dec 2016 13:18:41 -0000 --Apple-Mail=_8D94F912-B2C0-4356-BB86-195AC0E62D63 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, for Flink 1.2 the ListCheckpointed interface is intended to replace = Checkpointed. =46rom a user perspective they work very similar, but the = new interface allows you to break down the state into smaller, = redistributable units (the list items) to support job rescaling. A = proper documentation for the new checkpointing will be provided before = the 1.2 release. Best, Stefan > Am 14.12.2016 um 14:06 schrieb M=C3=A4ki Hanna = : >=20 > Hi,=20 > =20 > I=E2=80=99m learning Flink and trying to calculate counters that are = checkpointed at given intervals. Following the examples on page = https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/streaming= /state.html = , except in Scala, I tried to create a function > =20 > class CountersWithState extends FlatMapFunction[Event, String] with = Checkpointed[Counter] > =20 > but Checkpointed seems to be deprecated. What is the currently = preferred way to checkpoint instance fields? > =20 > I have also written a version that uses the flatMapWithState function, = but in my tests checkpoints are not created even though I have enabled = checkpointing using env.enableCheckpointing(10000). When I use = flatMapToState to calculate the state, do I have to enable checkpointing = somehow explicitly (in addition to env.enableCheckpointing)? > =20 > BR,=20 > Hanna=20 > Disclaimer: This message and any attachments thereto are intended = solely for the addressed recipient(s) and may contain confidential = information. If you are not the intended recipient, please notify the = sender by reply e-mail and delete the e-mail (including any attachments = thereto) without producing, distributing or retaining any copies = thereof. Any review, dissemination or other use of, or taking of any = action in reliance upon, this information by persons or entities other = than the intended recipient(s) is prohibited. Thank you. --Apple-Mail=_8D94F912-B2C0-4356-BB86-195AC0E62D63 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Hi,

for = Flink 1.2 the ListCheckpointed interface is intended to replace = Checkpointed. =46rom a user perspective they work very similar, but the = new interface allows you to break down the state into smaller, = redistributable units (the list items) to support job rescaling. A = proper documentation for the new checkpointing will be provided before = the 1.2 release.

Best,
Stefan

Am = 14.12.2016 um 14:06 schrieb M=C3=A4ki Hanna <Hanna.Maki@comptel.com>:

Hi, 
 
I=E2=80=99m learning Flink and trying to = calculate counters that are checkpointed at given intervals. Following = the examples on page https://ci.apache.org/projects/flink/flink-docs-release-1.1/api= s/streaming/state.html, except in Scala, I tried to create a = function
 
class CountersWithState = extends FlatMapFunction[Event, String] with Checkpointed[Counter]
 
but = Checkpointed seems to be deprecated. What is the currently preferred way = to checkpoint instance fields?
 
I have also written a version = that uses the flatMapWithState function, but in my tests checkpoints are = not created even though I have enabled checkpointing using = env.enableCheckpointing(10000). When I use flatMapToState to calculate = the state, do I have to enable checkpointing somehow explicitly (in = addition to env.enableCheckpointing)?
 
BR, 
Hanna 
Disclaimer: This = message and any attachments thereto are intended solely for the = addressed recipient(s) and may contain confidential information. If you = are not the intended recipient, please notify the sender by reply e-mail = and delete the e-mail (including any attachments thereto) without = producing, distributing or retaining any copies thereof. Any review, = dissemination or other use of, or taking of any action in reliance upon, = this information by persons or entities other than the intended = recipient(s) is prohibited. Thank = you.

= --Apple-Mail=_8D94F912-B2C0-4356-BB86-195AC0E62D63--