From user-return-3611-archive-asf-public=cust-asf.ponee.io@beam.apache.org Wed Jul 18 00:50:49 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 1360D180600 for ; Wed, 18 Jul 2018 00:50:48 +0200 (CEST) Received: (qmail 41193 invoked by uid 500); 17 Jul 2018 22:50:48 -0000 Mailing-List: contact user-help@beam.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@beam.apache.org Delivered-To: mailing list user@beam.apache.org Received: (qmail 41174 invoked by uid 99); 17 Jul 2018 22:50:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Jul 2018 22:50:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 817BF1A10F3; Tue, 17 Jul 2018 22:50:47 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.249 X-Spam-Level: ** X-Spam-Status: No, score=2.249 tagged_above=-999 required=6.31 tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.25, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id 5jHVblvtlCBc; Tue, 17 Jul 2018 22:50:46 +0000 (UTC) Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id B2B4D5F39F; Tue, 17 Jul 2018 22:50:45 +0000 (UTC) Received: by mail-lj1-f171.google.com with SMTP id v9-v6so2372920ljk.4; Tue, 17 Jul 2018 15:50:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Q/dAtn9c0i1HBvW+ZucO1iklOLUlGpHA3yqHOJve+gQ=; b=b8ciuxr1ebaP8HB74nQ6AbmUT3ab3hymRR17t3vAmNU0ZzUfMIa/b7NqORmqjKKR0p 21x9ffrtmFN9qG7/+P7fYPwzCpqmbRB2MHKK90Np2PjNm+83P7C6rH9jgkYBoMLvuQ6k M5NjB18Fmlg7eNR6ftqXM054Gs7IzOTxHbx+RhO9sn3f0V2L6qhW5JsO+0790EPXpsu5 Cx3VfLr6UnFd5OyYBAivacn9t4eHctjcVQIWmvKHX/SFdlSdGDLNChBFz3aQWHlMUG5Q 8EmOIEpNmHfJLM8FxxqZVkuKEi+oLA3XYf+u/KLB8mcwlHdLY6ghFqexqU54VywRkIbz zsFA== X-Gm-Message-State: AOUpUlGugsFUCjrIe1GPsj51z68uztukcPLIgrUDvnvP6Gp6dhNAh5WU Ucb4QrXCnFzycWwL8/fHVwPZenGLl48Zc6xhyM/dqg== X-Google-Smtp-Source: AAOMgpe7Y+GPoigyXdTNfVORcbuNg2qHytKpnrS1XHDIhjEeTn4FevYWSXj6cRa6yDcmqDb5eWGFztvmjkKbiY/fxMY= X-Received: by 2002:a2e:4701:: with SMTP id u1-v6mr2808522lja.54.1531867844343; Tue, 17 Jul 2018 15:50:44 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Holden Karau Date: Tue, 17 Jul 2018 15:50:32 -0700 Message-ID: Subject: Re: Live coding & reviewing adventures To: dev@beam.apache.org Cc: user@beam.apache.org Content-Type: multipart/alternative; boundary="000000000000b47a61057139c631" --000000000000b47a61057139c631 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Yup I=E2=80=99ve got a fork of that (I skip re-uploading the input data fil= es since conference WiFi) and store to gcs so I can see the output. Didn=E2=80=99t finish everything in today=E2=80=99s stream because the WiFi= was a bit flakey during the first container build and stalled apt, but I=E2=80=99ll d= o a follow up to finish it up. On Tue, Jul 17, 2018 at 2:34 PM Ankur Goenka wrote: > +1 > For reference here is a sample job > https://github.com/axelmagn/model-analysis/blob/axelmagn-hacks/examples/c= hicago_taxi/preprocess_flink.sh > > Also 1 quick heads up, output file will be created in docker container if > you use local file system. > > > > > On Tue, Jul 17, 2018 at 2:27 PM Holden Karau wrote= : > >> And I've got an hour to kill @ SFO today so at some of the suggestions >> from folks I'm going to do a more user focused one trying getting the TF= T >> demo to work with the portable flink runner (hopefully) - >> https://www.youtube.com/watch?v=3DwL9mvQeN36E >> >> On Fri, Jul 13, 2018 at 11:54 AM, Holden Karau >> wrote: >> >>> Hi folks! I've been doing some live coding in my other projects and I >>> figured I'd do some with Apache Beam as well. >>> >>> Today @ 3pm pacific I'm going be doing some impromptu exploration bette= r >>> review tooling possibilities (looking at forking spark-pr-dashboard for >>> other projects like beam and setting up mentionbot to work with ASF inf= ra) >>> - https://www.youtube.com/watch?v=3Dff8_jbzC8JI >>> >>> Next week (Thursday the 19th at 2pm pacific) I'm going to be working on >>> trying to get easier dependency management for the Python portable runn= er >>> in place - https://www.youtube.com/watch?v=3DSv0XhS2pYqA >>> >>> If your interested in seeing more of the development process I hope you >>> will join me :) >>> >>> P.S. >>> >>> You can also follow on twitch which does a better job of notifications >>> https://www.twitch.tv/holdenkarau >>> >>> Also one of the other thing I do is "live reviews" of PRs but they are >>> generally opt-in and I don't have enough opt-ins from the Beam communit= y to >>> do live reviews in Beam, if you work on Beam and would be OK with me do= ing >>> a live streamed review of your PRs let me know (if your curious to what >>> they look like you can see some of them here in Spark land >>> >>> ). >>> >>> -- >>> Twitter: https://twitter.com/holdenkarau >>> >> >> >> >> -- >> Twitter: https://twitter.com/holdenkarau >> > -- Twitter: https://twitter.com/holdenkarau --000000000000b47a61057139c631 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Yup I=E2=80=99ve got a fork of that (I skip re-uploa= ding the input data files since conference WiFi) and store to gcs so I can = see the output.

Di= dn=E2=80=99t finish everything in today=E2=80=99s stream because the WiFi w= as a bit flakey during the first container build and stalled apt, but I=E2= =80=99ll do a follow up to finish it up.

On Tue, Jul 17, 2018 at 2:34 PM Ankur Goenka <goenka@google.com> wrote:
+1=C2=A0
Also 1 quick= heads up, output file will be created in docker container if you use local= file system.




<= div class=3D"gmail_quote">
On Tue, Jul 17, 2018 at 2:27 PM = Holden Karau <= holden@pigscanfly.ca> wrote:
And I've got an hour to kill @ SFO today so at some o= f the suggestions from folks I'm going to do a more user focused one tr= ying getting the TFT demo to work with the portable flink runner (hopefully= ) -=C2=A0https://www.youtube.com/watch?v=3DwL9mvQeN36E

On Fri, Jul 13, 2018 at 1= 1:54 AM, Holden Karau <holden@pigscanfly.ca> wrote:
Hi folks! I've been doing= some live coding in my other projects and I figured I'd do some with A= pache Beam as well.

Today @ 3pm pacific I'm going be= doing some impromptu exploration better review tooling possibilities (look= ing at forking spark-pr-dashboard for other projects like beam and setting = up mentionbot to work with ASF infra) -=C2=A0https://www.youtube.com/watch= ?v=3Dff8_jbzC8JI

Next week (Thursday the 19th = at 2pm pacific) I'm going to be working on trying to get easier depende= ncy management for the Python portable runner in place -=C2=A0https://www.= youtube.com/watch?v=3DSv0XhS2pYqA

If your inte= rested in seeing more of the development process I hope you will join me :)=

P.S.

You can also follow= on twitch which does a better job of notifications=C2=A0https://www.twitch.tv/holdenk= arau

Also one of the other thing I do is "= ;live reviews" of PRs but they are generally opt-in and I don't ha= ve enough opt-ins from the Beam community to do live reviews in Beam, if yo= u work on Beam and would be OK with me doing a live streamed review of your= PRs let me know (if your curious to what they look like you can see some o= f them here= in Spark land).
=

--



--
--
<= div dir=3D"ltr">
--000000000000b47a61057139c631--