From dev-return-8524-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Thu May 23 06:48:53 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 04A7518062F for ; Thu, 23 May 2019 08:48:52 +0200 (CEST) Received: (qmail 77536 invoked by uid 500); 23 May 2019 06:48:48 -0000 Mailing-List: contact dev-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list dev@airflow.apache.org Received: (qmail 77508 invoked by uid 99); 23 May 2019 06:48:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 May 2019 06:48:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 180B9182668 for ; Thu, 23 May 2019 06:48:47 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -13.7 X-Spam-Level: X-Spam-Status: No, score=-13.7 tagged_above=-999 required=6.31 tests=[DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, ENV_AND_HDR_SPF_MATCH=-0.5, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5, USER_IN_DEF_SPF_WL=-7.5] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=google.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id 8-RS7I8KAEp9 for ; Thu, 23 May 2019 06:48:45 +0000 (UTC) Received: from mail-it1-f174.google.com (mail-it1-f174.google.com [209.85.166.174]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id B3C41612BD for ; Thu, 23 May 2019 06:31:16 +0000 (UTC) Received: by mail-it1-f174.google.com with SMTP id 9so6868755itf.4 for ; Wed, 22 May 2019 23:31:16 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=D5O/Nax1PdSuHCXE5fezbi9JlaXOmxkvKsy9QkrXBr0=; b=TEnhWPtTCWhmir3Jqd87p4S+ffQ/C+U3eZi4Eaf+/GHQPBlMKX5lyQai07JJgRZmIf 8Rctz6ClYfSCjNOAzMMLSfFONfLndTf+OEYGqsqzQKo/k8V5nll8Fl0x8Qed+HJKjb+L dzWUwcXghdDB1QlSAeyAUZINHFEESZSnAmkZTt6j9cOXmYJ7voIhCT1xjcPyKR6zZr/e VXECv+pC4O9Cu1b5LRlItQdFds1Erd5wflTapN6rCPeqPYxCX7xqdX2/RvZe0laqGx98 DPy2af+YFBnMwN7bb5tnjykPUsGGFuSHJ8b75lRaPmb+bJ2/ei/xQk4buZ9lkjF/k8u5 xWrQ== X-Gm-Message-State: APjAAAW26aPPUf76UfAWYbdn4G2pm5KCS7cB8dGJ15kqSofLVK+gHYHw u0qOsQsRC3ekHyk/TznJVeVPQ9hpjw0UfCYWsjN44l5vW2o= X-Google-Smtp-Source: APXvYqwgUXvmxQdYV7tTxpQRWvsz2UYEJmwfcmdRO2nxj5dBK5yXbyYYJM7thH/IB8jc34wxd91KxWY3+mdp0qvGjaA= X-Received: by 2002:a24:dac7:: with SMTP id z190mr11852462itg.57.1558593075571; Wed, 22 May 2019 23:31:15 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Feng Lu Date: Wed, 22 May 2019 23:31:04 -0700 Message-ID: Subject: Re: Bay Area Airflow meetup To: dev@airflow.apache.org Cc: Griselda Cuevas , Leah Cole , Trishka Fernandes Content-Type: multipart/alternative; boundary="0000000000009ed8440589883a2c" --0000000000009ed8440589883a2c Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable We have now finalized the meetup schedule and logistics, lots of interesting talks, please check out the details here. Also a big thank you to all the presenters, see you soon! Feng On Fri, May 10, 2019 at 3:54 PM Feng Lu wrote: > Great, thank you Gabriel. > > *From: *Gabriel Silk > *Date: *Fri, May 10, 2019 at 1:44 PM > *To: * > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes > > Sure, how about: >> >> "Migrating Dropbox to Airflow, and the challenges of ETL at scale" >> >> Abstract: Dropbox has over 20k batch jobs that run on an hourly and/or >> daily cadence, connected together in a large web of dependencies. >> Migrating >> this ETL to Airflow has proven to be an interesting challenge. In this >> talk >> we'll discover how we went about it, what challenges we are facing, how >> Airflow is helping us, and opportunities we see ahead" >> >> On Fri, May 10, 2019 at 1:05 PM Feng Lu >> wrote: >> >> > Hi Gabriel, >> > >> > Awesome, could you share with us the talk title and abstract? >> > Thanks. >> > >> > Feng >> > >> > *From: *Gabriel Silk >> > *Date: *Fri, May 10, 2019 at 10:15 AM >> > *To: * >> > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes >> > >> > Hey Feng, >> > > >> > > I'm the tech lead on Data Infrastructure at Dropbox, and I'd like to >> talk >> > > about the challenges and opportunities in migrating large-scale ETL = to >> > > Airflow from a system that was built in-house. >> > > >> > > >> > > On Fri, May 10, 2019 at 8:23 AM Szymon Przedwojski < >> > > szymon.przedwojski@polidea.com> wrote: >> > > >> > > > Hi Feng, >> > > > >> > > > I'd like to submit my talk proposal for the Bay Area meetup. >> > > > >> > > > *Working title:* Migrating Apache Oozie Workflows to Apache Airflo= w >> - >> > the >> > > > developer=E2=80=99s perspective >> > > > >> > > > I want to talk about the Oozie-to-Airflow [1] project created by >> Google >> > > > that we at Polidea help develop. >> > > > It's a conversion tool written in Python that generates Airflow >> Python >> > > DAGs >> > > > from Oozie workflow XMLs, thus enabling easy migration of Oozie >> > workflows >> > > > to Airflow. >> > > > I would like to present the architecture of the solution and of th= e >> > whole >> > > > end-to-end system tests suite, talk about the challenges we faced >> when >> > > > translating various Oozie features to Airflow and discuss the >> future of >> > > the >> > > > project. >> > > > Moreover, I will show how the Airflow community can get involved i= n >> the >> > > > open-source Oozie-to-Airflow project and why it would be mutually >> > > > beneficial. >> > > > >> > > > *About me:* >> > > > I=E2=80=99m a Software Engineer @ Polidea in Warsaw. I=E2=80=99ve = been programming >> > > > professionally for 5 years now, mainly using Python, Java and JS. >> > > > I have public speaking experience - I gave a couple of talks at >> Python >> > > and >> > > > JS meetups, at a GDG Dev Fest Warsaw event and multiple internal >> talks >> > at >> > > > companies I=E2=80=99ve worked for. >> > > > I=E2=80=99ve been involved with Airflow since August last year and= I=E2=80=99ve >> > > contributed >> > > > several operators for different GCP services. >> > > > >> > > > [1] https://github.com/GoogleCloudPlatform/cloud-composer >> > > > >> > > > *Szymon Przedwojski* >> > > > Polidea | Software Engineer >> > > > >> > > > M: +48 500 330 790 >> > > > >> > > > >> > > > On Thu, 18 Apr 2019 at 01:08, Feng Lu >> > wrote: >> > > > >> > > > > We've finalized the meetup logistics (details here >> > > > > < >> > > > > >> > > > >> > > >> > >> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/= 260712102/ >> > > > > >). >> > > > > Please send in your talk proposals! >> > > > > Thank you. >> > > > > >> > > > > Feng >> > > > > >> > > > > On Tue, Mar 12, 2019 at 1:15 PM Feng Lu >> wrote: >> > > > > >> > > > > > Hi all, >> > > > > > >> > > > > > It has been a while since we had the last Bay Area meetup and = we >> > > > (Google) >> > > > > > are going to host another meetup around June this year. Will >> share >> > > with >> > > > > > everyone the logistic details, meanwhile, please feel free to >> send >> > in >> > > > > your >> > > > > > talk proposals! >> > > > > > (I see a lot of very interesting AIP ideas floating around, >> these >> > are >> > > > > > great candidates.) >> > > > > > >> > > > > > Looking forward to seeing you all at the event ;) >> > > > > > >> > > > > > Feng >> > > > > > >> > > > > >> > > > >> > > >> > >> > --0000000000009ed8440589883a2c--