From dev-return-17219-archive-asf-public=cust-asf.ponee.io@beam.apache.org Sat Jun 1 01:21:22 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id EEC3B180627 for ; Sat, 1 Jun 2019 03:21:21 +0200 (CEST) Received: (qmail 53446 invoked by uid 500); 1 Jun 2019 01:21:20 -0000 Mailing-List: contact dev-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@beam.apache.org Delivered-To: mailing list dev@beam.apache.org Received: (qmail 53436 invoked by uid 99); 1 Jun 2019 01:21:20 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 01 Jun 2019 01:21:20 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id C455018106D for ; Sat, 1 Jun 2019 01:21:19 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -13.7 X-Spam-Level: X-Spam-Status: No, score=-13.7 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, T_KAM_HTML_FONT_INVALID=0.01, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id fl-6Kf4o6Arp for ; Sat, 1 Jun 2019 01:21:18 +0000 (UTC) Received: from mail-lj1-f180.google.com (mail-lj1-f180.google.com [209.85.208.180]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id B6CC55F16F for ; Sat, 1 Jun 2019 01:21:17 +0000 (UTC) Received: by mail-lj1-f180.google.com with SMTP id j24so11299250ljg.1 for ; Fri, 31 May 2019 18:21:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=2TacKWXoiME5/v7Nb66IKpHJQOclVurdGbZPpa8iXj8=; b=OUdG8lq9esgfzOwK8qNwW/1bHnuC+jFYeon28DG3SEYolE3ERsV/ZS8zW2zKPwzrjb 3Fvxx+3c1CZx0o17fnWZlTAbmghnyzNTsDuZIADus2lQ1+y0bdPgixTF4XQ7RoIeC0Kn DS7nAuCfOm1euZc+BjD8UGMPviWMBW9zIRZ+gZeDeFbTVajSmsF99ReE7p+15Q2h9bKe Dbah9Y4DriVpwHg7dUITo+bPh69E2phivXBA2m9D7gKam5QNJY10LdGSXSzGyDvj3Kkw q+wdCVARepuHRMPkgevmMIifd4LykY6kYFqeItifx551M/MYlOcwmogMOXnSPKTIB1fa QSlg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=2TacKWXoiME5/v7Nb66IKpHJQOclVurdGbZPpa8iXj8=; b=Xj9hPTbIeBUQ4/U0pQ9zU25PB3RZtBBBh/MyuD7gV8d5uk0oViL8vsmnzDgCPdITBm jWZwo0p8+Tg4ocPjgsCfvDIRUItYt5pcRkmeQOsmWNEYvjfhxUoOID/g+78mWrO00Vnj 8DT07rBi/vE/FHtEQTbl+oWp2qPazCJ7aB3EpLU+AJ1qQ6wSF6/ONodcvljJ0M6KCFQH 5NzGPCyA6/zD8MGPvh/9WPzhTeK5regm9kgKs0RhSlZF9uTqhbdve+Xrgxxn7+i0lxux opnMpTxcSYzx3dE5DlzmvXbiOspCkc5vmku/OlvMAeUhyNdFREJGg4W3v1aCIQyCKmPA Av/w== X-Gm-Message-State: APjAAAU4FWCoF9ALLesHUsJbSYEvefU206oliMFfRiZAXy0QB2tvhISg WnivUA4UF1e0jSXRilXf2k2Qr3tcP3U9hNX+kzl9Wybmml8= X-Google-Smtp-Source: APXvYqx8f/8afzIqLuwIk4wDGc6mpOpjQCBvnGhHDw5rC6C/mHQhGFwve+uhGeBXW/Sz/Kgs+VsgZ0qQb4Cj6qrTYg0= X-Received: by 2002:a2e:568b:: with SMTP id k11mr7652656lje.124.1559352075729; Fri, 31 May 2019 18:21:15 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ahmet Altay Date: Fri, 31 May 2019 18:21:04 -0700 Message-ID: Subject: Re: [DISCUSS] Cookbooks for users with knowledge in other frameworks To: dev Content-Type: multipart/alternative; boundary="0000000000008e20d3058a38f2d0" --0000000000008e20d3058a38f2d0 Content-Type: text/plain; charset="UTF-8" Thank you Reza. That separation makes sense to me. On Wed, May 29, 2019 at 6:26 PM Reza Rokni wrote: > +1 > > I think there will be at least two layers of this; > > Layer 1 - Using primitives : I do join, GBK, Aggregation... with system x > this way, what is the canonical equivalent in Beam. > Layer 2 - Patterns : I read and join Unbounded and Bounded Data in system > x this way, what is the canonical equivalent in Beam. > > I suspect as a first pass Layer 1 is reasonably well bounded work, there > would need to be agreement on "canonical" version of how to do something in > Beam as this could be seen to be opinionated. As there are often a > multitude of ways of doing x.... > Once we identify a set of layer 1 items, we could crowd source the canonical implementations. I believe we can use our usual code review process to settle on a version that is agreeable. (Examples have the same issue, they are probably opinionated today based on the author but it works out.) > > > On Thu, 30 May 2019 at 08:56, Ahmet Altay wrote: > >> Hi all, >> >> Inspired by the user asking about a Spark feature in Beam [1] in the >> release thread, I searched the user@ list and noticed a few instances of >> people asking for question like "I can do X in Spark, how can I do that in >> Beam?" Would it make sense to add documentation to explain how certain >> tasks that can be accomplished in Beam with side by side examples of doing >> the same task in Beam/Spark etc. It could help with on-boarding because it >> will be easier for people to leverage their existing knowledge. It could >> also help other frameworks as well, because it will serve as a Rosetta >> stone with two translations. >> >> Questions I have are: >> - Would such a thing be a helpful? >> - Is it feasible? Would a few pages worth of examples can cover enough >> use cases? >> >> Thank you! >> Ahmet >> >> [1] >> https://lists.apache.org/thread.html/b73a54aa1e6e9933628f177b04a8f907c26cac854745fa081c478eff@%3Cdev.beam.apache.org%3E >> > > > -- > > This email may be confidential and privileged. If you received this > communication by mistake, please don't forward it to anyone else, please > erase all copies and attachments, and please let me know that it has gone > to the wrong person. > > The above terms reflect a potential business arrangement, are provided > solely as a basis for further discussion, and are not intended to be and do > not constitute a legally binding obligation. No legally binding obligations > will be created, implied, or inferred until an agreement in final form is > executed in writing by all parties involved. > --0000000000008e20d3058a38f2d0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thank you Reza. That separation makes sen= se to me.

On Wed, May 29, 2019 at 6:26 PM Reza Rokni <rez@google.com> wrote:
+1
I think there will be at least two layers of this;
Layer 1 - Using primitives : I do join, GBK, Aggregation... wi= th system x this way, what is the canonical equivalent=C2=A0in Beam.=C2=A0 = =C2=A0
Layer 2 - Patterns : I read and join Unbounded and Bounded= Data in system x this way,=C2=A0what is the canonical equivalent=C2=A0in B= eam.=C2=A0 =C2=A0

I suspect as a first pass Layer = 1 is reasonably well bounded work, there would need to be agreement on &quo= t;canonical" version of how to do something in Beam as this could be s= een to be opinionated. As there are often a multitude of ways of doing x...= .=C2=A0

Once we identify a set = of layer 1 items, we could crowd source the canonical implementations. I be= lieve we can use our usual code review process to settle on a version that = is agreeable. (Examples have the same issue, they are probably=C2=A0opinion= ated today based on the author but it works out.)
=C2=A0


On Thu, 30 May 2019 at 08:56, Ahmet Altay <altay@google.com> wrote:
Hi all,
Inspired by the user asking about a Spark feature in Beam [1] in the rele= ase thread, I searched the user@ list and noticed a few instances of people= asking for question like "I can do X in Spark, how can I do that in B= eam?" Would it make sense to add documentation to explain how certain = tasks that can be accomplished in Beam with side by side examples of doing = the same task in Beam/Spark etc. It could help with on-boarding because it = will be easier for people to leverage their existing knowledge. It could al= so help other frameworks as well, because it will serve as a Rosetta stone = with two translations.

Questions I have are:
- Would such a thing= be a helpful?
- Is it feasible? Would a few pages worth of examples can= cover enough use cases?

Thank you!
Ahmet



--

This email may be confidential and privileged. I= f you received this communication by mistake, please don't forward it t= o anyone else, please erase all copies and attachments, and please let me k= now that it has gone to the wrong person.

The above terms reflect a = potential business arrangement, are provided solely as a basis for further = discussion, and are not intended to be and do not constitute a legally bind= ing obligation. No legally binding obligations will be created, implied, or= inferred until an agreement in final form is executed in writing by all pa= rties involved.

--0000000000008e20d3058a38f2d0--