From dev-return-525-archive-asf-public=cust-asf.ponee.io@heron.incubator.apache.org Mon Apr 2 09:02:31 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 0B743180627 for ; Mon, 2 Apr 2018 09:02:30 +0200 (CEST) Received: (qmail 87462 invoked by uid 500); 2 Apr 2018 07:02:30 -0000 Mailing-List: contact dev-help@heron.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@heron.incubator.apache.org Delivered-To: mailing list dev@heron.incubator.apache.org Received: (qmail 87446 invoked by uid 99); 2 Apr 2018 07:02:29 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 02 Apr 2018 07:02:29 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id F3AA41A0311 for ; Mon, 2 Apr 2018 07:02:28 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.879 X-Spam-Level: * X-Spam-Status: No, score=1.879 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 8HbObK4gBVXY for ; Mon, 2 Apr 2018 07:02:26 +0000 (UTC) Received: from mail-it0-f47.google.com (mail-it0-f47.google.com [209.85.214.47]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id E0D6A5F260 for ; Mon, 2 Apr 2018 07:02:25 +0000 (UTC) Received: by mail-it0-f47.google.com with SMTP id f6-v6so6572306ita.2 for ; Mon, 02 Apr 2018 00:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=WJeD/67DWko7zdPAC00ZgSdgfAUUUoVgCSMX31eBXcQ=; b=mzrFl1vqPEfgsm7kwF6xgmvcwifnWMWb2j2LgUq7n1L5axMCsLrcEulZrEgCtJ0scQ w0oyHQYQWDb/ztPVKf0xl0C6zkxQOWZifUZOKAm7c9Jl17V0hQinM6v0dZJ166SPiPLZ 9rBz/E1ojjYVgJ5LwDYg+ziR8uvsOByqBwfhRShnvMYnOTmxoYJ7Oj834nCqWWComEsN 4xoxVDQn85CWPXbuFLM7/KyOBovepoDwQKJhGN82/mSDO/bM+7WwusSkRLja3u02B2WP aO5w22iLKqP//YqZ40l9bmOEZMCEyVoavRP8nIz1UDWapLzu62MAUm4xUNQNx86L1thh Z/fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=WJeD/67DWko7zdPAC00ZgSdgfAUUUoVgCSMX31eBXcQ=; b=P1S24bDc/Lyu8PnLev3N/7/xASbe+iPINpkiWp0zFEpdCtxXQOv/J0Son8S1F4hKqZ 728B5nl2/Nb8TooAxsKcToMpDODIHJOJ/RHvuXngTZFFzzHC//bnNeSI4vDKf3hkKMCZ FqmE502o1BDKUOxVdCQVImDGA0Qrl3IAZv5NVOrWHRTKsrOIGogMR7AOJ2ONv5bar+yu A5vGHO6qR3SqY41oCzBXsmpLCkdI/905f2XMFJjP5o/40BLYJiwn+d11zHcLlI24cy5b 4PUFp853OPnMpSlcsl1uNNxbCXAI3rMTGJ5UjYJYEyMOLnUKE18NAauROnXdqrzdjtRQ ZkhA== X-Gm-Message-State: ALQs6tCkk9x5APkWbJZZM7Ujluw6aYr1dY4YckySDCyHANYgRydqlYbG FkwcbGKNZ/yWndychLB/01JOahV1rP7J+t3w819s2g== X-Google-Smtp-Source: AIpwx4/fNxUa2kdyVMTKVbku7cY//1FI4TEQC6S5GcEDke/NizhSwUOPVa0qsIuOPt/1dCmpwRnG1h+vB02k87Lbw5s= X-Received: by 2002:a24:67d7:: with SMTP id u206-v6mr11341571itc.138.1522652544301; Mon, 02 Apr 2018 00:02:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.12.154 with HTTP; Mon, 2 Apr 2018 00:02:23 -0700 (PDT) In-Reply-To: References: <4F998B1B-1B0A-4FEF-8761-0D20EFCE6322@gmail.com> From: Ning Wang Date: Mon, 2 Apr 2018 00:02:23 -0700 Message-ID: Subject: Re: Stateful Updates To: dev@heron.incubator.apache.org Content-Type: multipart/alternative; boundary="00000000000004fed30568d82c00" --00000000000004fed30568d82c00 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable I think it is a draft currently and it might be easier to collaborate in Google Doc. On Sun, Apr 1, 2018 at 10:49 PM, Dave Fisher wrote: > Hi - > > Any reason this document couldn=E2=80=99t move to either thus mailing lis= t or an > issue or wiki on github? > > Regards, > Dave > > > On Apr 1, 2018, at 10:38 PM, Karthik Ramasamy > wrote: > > > > Tom - Added a bunch of feedback > > > > cheers > > /karthik > > > >> On Sun, Apr 1, 2018 at 3:24 PM, Karthik Ramasamy > wrote: > >> > >> Tom - permissions look good now. > >> > >> On Sun, Apr 1, 2018 at 3:03 PM Thomas Cooper > >> wrote: > >> > >>> Hi all, > >>> > >>> Hopefully the share permissions are correct on the doc now: > >>> > >>> > >>> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt > >>> i7Q0hmW6lmgyk8/edit?usp=3Dsharing > >>> > >>> Look forward to hearing what you think. > >>> > >>> Also is it worth making an issue for this? > >>> > >>> Cheers, > >>> > >>> Tom Cooper > >>> W: www.tomcooper.org.uk | Twitter: @tomncooper > >>> > >>> > >>>> On 30 March 2018 at 20:05, Karthik Ramasamy > wrote: > >>>> > >>>> Thanks Tom. It will be a great addition. Unable to access the doc. > Could > >>>> you give permission? > >>>> > >>>> Sent from my iPhone > >>>> > >>>>> On Mar 30, 2018, at 6:08 PM, Thomas Cooper > >>>> wrote: > >>>>> > >>>>> Hi all, > >>>>> > >>>>> After chatting to the Heron Team at Twitter and Microsoft I have > >>> written > >>>> up > >>>>> a proposal for giving Heron the ability to preserve state across a > >>>> topology > >>>>> update (change in component parallelism). This is problem for > >>> components > >>>>> with incoming fields based connections as keys will not hash to the > >>> same > >>>>> task ID after an update. I have defined the problem (as I see it) a= nd > >>>>> propsoed two solutions in the google doc below: > >>>>> > >>>>> https://docs.google.com/document/d/1X0pS9uwevn16nYYqrGjEvWQMWQLwt > >>>> i7Q0hmW6lmgyk8/edit?usp=3Dsharing > >>>>> > >>>>> I would love to get feedback on these proposals. Feel free to blow > >>> holes > >>>> in > >>>>> my assumptions and correct any misinterpretations I have made! > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Tom Cooper > >>>>> W: www.tomcooper.org.uk | Twitter: @tomncooper > >>>>> > >>>> > >>> > >> > > --00000000000004fed30568d82c00--