Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 2B127200C1B for ; Tue, 14 Feb 2017 08:31:21 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 29B38160B5F; Tue, 14 Feb 2017 07:31:21 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 717FA160B52 for ; Tue, 14 Feb 2017 08:31:20 +0100 (CET) Received: (qmail 2126 invoked by uid 500); 14 Feb 2017 07:31:19 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 2112 invoked by uid 99); 14 Feb 2017 07:31:19 -0000 Received: from mail-relay.apache.org (HELO mail-relay.apache.org) (140.211.11.15) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Feb 2017 07:31:19 +0000 Received: from MacBook.mail (36-230-68-112.dynamic-ip.hinet.net [36.230.68.112]) by mail-relay.apache.org (ASF Mail Server at mail-relay.apache.org) with ESMTPSA id 546241A0015; Tue, 14 Feb 2017 07:31:18 +0000 (UTC) Date: Tue, 14 Feb 2017 15:31:14 +0800 From: "Tzu-Li (Gordon) Tai" To: user@flink.apache.org, s.richter@data-artisans.com, srichter@apache.org Message-ID: In-Reply-To: References: Subject: Re: A way to control redistribution of operator state? X-Mailer: Airmail (397) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="58a2b242_1083a46b_263" archived-at: Tue, 14 Feb 2017 07:31:21 -0000 --58a2b242_1083a46b_263 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi Dmitry, Technically, from the looks of the internal code around =60OperatorStateR= epartitioner=60, I think it is certainly possible to be pluggable. Right now it is just hard coded to use a round-robin repartitioner implem= entation as default. However, I=E2=80=99m not sure of the plans in exposing this to the user a= nd making it configurable. Looping in Stefan (in cc) who mostly worked on this part and see if he ca= n provide more info. - Gordon On =46ebruary 14, 2017 at 2:30:27 AM, Dmitry Golubets (dgolubets=40gmail.= com) wrote: Hi, It looks impossible to implement a keyed state with operator state now. I know it sounds like =22just use a keyed state=22, but latter requires u= pdating it on every value change as opposed to operator state and thus ca= n be expensive (especially if you have to deal with mutable structures in= side which have to be serialized). The problem is that there is no way to tell =46link how to reassign savep= oint parts between partitions, and thus impossible to route data to corre= ct partitions. Is there anything I missed or maybe a plan to implement it in future=3F Best regards, Dmitry --58a2b242_1083a46b_263 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline