From dev-return-8611-archive-asf-public=cust-asf.ponee.io@airflow.apache.org Tue Jun 4 22:59:43 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 3A4E218064D for ; Wed, 5 Jun 2019 00:59:43 +0200 (CEST) Received: (qmail 67635 invoked by uid 500); 4 Jun 2019 22:59:41 -0000 Mailing-List: contact dev-help@airflow.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@airflow.apache.org Delivered-To: mailing list dev@airflow.apache.org Received: (qmail 67604 invoked by uid 99); 4 Jun 2019 22:59:41 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 04 Jun 2019 22:59:41 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 74514C63A6 for ; Tue, 4 Jun 2019 22:59:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.001 X-Spam-Level: ** X-Spam-Status: No, score=2.001 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=flyrlabs-com.20150623.gappssmtp.com Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id DxJXeyMSBzw1 for ; Tue, 4 Jun 2019 22:59:37 +0000 (UTC) Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id F33935F17E for ; Tue, 4 Jun 2019 22:59:36 +0000 (UTC) Received: by mail-ed1-f53.google.com with SMTP id g13so2841935edu.5 for ; Tue, 04 Jun 2019 15:59:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=flyrlabs-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=JmyZTmrMRCq2SmZ37BGHzllTr9WAvTLIw6G/D6MP7cY=; b=IGiH7EFg+m2MdlbMy9ODt50x1A0JHvCx00+QAFsmaLuEQfKXMm2w0vOdARFUhq/NCZ UYkPnchBa8krYsZ+HBxJ1JWkbjgl/UF6fzqy7LeLQmEzi9OhctOP4/8G9EbJs6bGImqG mm59hxHA1gc+T9j7+jYmwQ3jVjWpNUyyNMXwXFdcWpPm36raAb394uZXJY0WhD+6QfUX rybOkazrgSbQLTIo8qGU2tOMfxRJq5Ss4Ky7bzL7T3bHRibeWpuOIo1CxTy1ExMZO70W 9Ai9pIbj24NHapEm17riDg7Tr/HtG9EMRPo0ocvNB70ScErnz2c1YnJkHeo8Qic+S8Xa vdow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=JmyZTmrMRCq2SmZ37BGHzllTr9WAvTLIw6G/D6MP7cY=; b=QBQDwekFWeezmRWirudGK4nj1JlLH/a+9xjpqs1H91sO2+YXWWjljLkPo4EZMUFTxx gpI7lSsC0QIrwpgtQ2AXcSczWR/9JLdd23ZvV9781+K8r1QB1mp8DSfQXfpuWnk3r2m8 Ik04Qeip9L4SHcbiBh+b9ifEPKo80JAQePyEdNhRK6t/jQVRhrrvRE+Tm8Cji5RoZV3H CZuAXhEfkXnjQ1sdKkHHqhwo1Fombz2B/8GstNte+60nveHzKxGhw+FXQYRpY4iJ5LN4 W6uKOJDQAzEDqcgOTjk92kCAUiZdOX7tjsSKSMHfTnpruEseW61z+/TNpU6Wop/v1caO G7/w== X-Gm-Message-State: APjAAAXdH7wnCHp1BHx4mz3ve8CNJEa18NItR7RPwbbvmtq+Iy+uJzTj j+H+1662stdHS8mR4gpEy0BeK5yKdKwzE5VqxST+rlCD X-Google-Smtp-Source: APXvYqzF4V5Lq9PSNlkIQlQrbH0ROBbWyKJNMESRkpdi0SEKHk7FDMbcXpCwhZ8seaZcmU/BdqmYDOosRFqpwPnDOOY= X-Received: by 2002:a50:90c6:: with SMTP id d6mr17769855eda.19.1559689176538; Tue, 04 Jun 2019 15:59:36 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Austin Weaver Date: Tue, 4 Jun 2019 15:59:25 -0700 Message-ID: Subject: Re: Bay Area Airflow meetup To: dev@airflow.apache.org Cc: Griselda Cuevas , Leah Cole , Trishka Fernandes Content-Type: multipart/alternative; boundary="00000000000053f15d058a876f17" --00000000000053f15d058a876f17 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for the reminder! Unfortunately the same time as the warriors are playing the nba finals in oakland ;( Is there any chance of this being recorded? Thanks! Austin On Tue, Jun 4, 2019 at 3:40 PM Feng Lu wrote: > A gentle reminder, the meetup will be tomorrow and we just updated the > event > location > < > https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/2= 60712102/ > > > to accommodate more people. > See you there. > > On Thu, May 23, 2019 at 11:08 AM Aizhamal Nurmamat kyzy > wrote: > > > Great list of speakers and topics! Will there be a live-streaming? > > > > Looking forward to the talks :) > > > > On Wed, May 22, 2019 at 11:48 PM Feng Lu > > wrote: > > > > > We have now finalized the meetup schedule and logistics, lots of > > > interesting talks, please check out the details > > > < > > > > > > https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/2= 60712102/ > > > > > > > here. > > > Also a big thank you to all the presenters, see you soon! > > > > > > Feng > > > > > > On Fri, May 10, 2019 at 3:54 PM Feng Lu wrote: > > > > > > > Great, thank you Gabriel. > > > > > > > > *From: *Gabriel Silk > > > > *Date: *Fri, May 10, 2019 at 1:44 PM > > > > *To: * > > > > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes > > > > > > > > Sure, how about: > > > >> > > > >> "Migrating Dropbox to Airflow, and the challenges of ETL at scale" > > > >> > > > >> Abstract: Dropbox has over 20k batch jobs that run on an hourly > and/or > > > >> daily cadence, connected together in a large web of dependencies. > > > >> Migrating > > > >> this ETL to Airflow has proven to be an interesting challenge. In > this > > > >> talk > > > >> we'll discover how we went about it, what challenges we are facing= , > > how > > > >> Airflow is helping us, and opportunities we see ahead" > > > >> > > > >> On Fri, May 10, 2019 at 1:05 PM Feng Lu > > > >> wrote: > > > >> > > > >> > Hi Gabriel, > > > >> > > > > >> > Awesome, could you share with us the talk title and abstract? > > > >> > Thanks. > > > >> > > > > >> > Feng > > > >> > > > > >> > *From: *Gabriel Silk > > > >> > *Date: *Fri, May 10, 2019 at 10:15 AM > > > >> > *To: * > > > >> > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes > > > >> > > > > >> > Hey Feng, > > > >> > > > > > >> > > I'm the tech lead on Data Infrastructure at Dropbox, and I'd > like > > to > > > >> talk > > > >> > > about the challenges and opportunities in migrating large-scal= e > > ETL > > > to > > > >> > > Airflow from a system that was built in-house. > > > >> > > > > > >> > > > > > >> > > On Fri, May 10, 2019 at 8:23 AM Szymon Przedwojski < > > > >> > > szymon.przedwojski@polidea.com> wrote: > > > >> > > > > > >> > > > Hi Feng, > > > >> > > > > > > >> > > > I'd like to submit my talk proposal for the Bay Area meetup. > > > >> > > > > > > >> > > > *Working title:* Migrating Apache Oozie Workflows to Apache > > > Airflow > > > >> - > > > >> > the > > > >> > > > developer=E2=80=99s perspective > > > >> > > > > > > >> > > > I want to talk about the Oozie-to-Airflow [1] project create= d > by > > > >> Google > > > >> > > > that we at Polidea help develop. > > > >> > > > It's a conversion tool written in Python that generates > Airflow > > > >> Python > > > >> > > DAGs > > > >> > > > from Oozie workflow XMLs, thus enabling easy migration of > Oozie > > > >> > workflows > > > >> > > > to Airflow. > > > >> > > > I would like to present the architecture of the solution and > of > > > the > > > >> > whole > > > >> > > > end-to-end system tests suite, talk about the challenges we > > faced > > > >> when > > > >> > > > translating various Oozie features to Airflow and discuss th= e > > > >> future of > > > >> > > the > > > >> > > > project. > > > >> > > > Moreover, I will show how the Airflow community can get > involved > > > in > > > >> the > > > >> > > > open-source Oozie-to-Airflow project and why it would be > > mutually > > > >> > > > beneficial. > > > >> > > > > > > >> > > > *About me:* > > > >> > > > I=E2=80=99m a Software Engineer @ Polidea in Warsaw. I=E2=80= =99ve been > > programming > > > >> > > > professionally for 5 years now, mainly using Python, Java an= d > > JS. > > > >> > > > I have public speaking experience - I gave a couple of talks > at > > > >> Python > > > >> > > and > > > >> > > > JS meetups, at a GDG Dev Fest Warsaw event and multiple > internal > > > >> talks > > > >> > at > > > >> > > > companies I=E2=80=99ve worked for. > > > >> > > > I=E2=80=99ve been involved with Airflow since August last ye= ar and > I=E2=80=99ve > > > >> > > contributed > > > >> > > > several operators for different GCP services. > > > >> > > > > > > >> > > > [1] https://github.com/GoogleCloudPlatform/cloud-composer > > > >> > > > > > > >> > > > *Szymon Przedwojski* > > > >> > > > Polidea | Software Engineer > > > >> > > > > > > >> > > > M: +48 500 330 790 > > > >> > > > > > > >> > > > > > > >> > > > On Thu, 18 Apr 2019 at 01:08, Feng Lu > > > > > > >> > wrote: > > > >> > > > > > > >> > > > > We've finalized the meetup logistics (details here > > > >> > > > > < > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/2= 60712102/ > > > >> > > > > >). > > > >> > > > > Please send in your talk proposals! > > > >> > > > > Thank you. > > > >> > > > > > > > >> > > > > Feng > > > >> > > > > > > > >> > > > > On Tue, Mar 12, 2019 at 1:15 PM Feng Lu > > > >> wrote: > > > >> > > > > > > > >> > > > > > Hi all, > > > >> > > > > > > > > >> > > > > > It has been a while since we had the last Bay Area meetu= p > > and > > > we > > > >> > > > (Google) > > > >> > > > > > are going to host another meetup around June this year. > Will > > > >> share > > > >> > > with > > > >> > > > > > everyone the logistic details, meanwhile, please feel fr= ee > > to > > > >> send > > > >> > in > > > >> > > > > your > > > >> > > > > > talk proposals! > > > >> > > > > > (I see a lot of very interesting AIP ideas floating > around, > > > >> these > > > >> > are > > > >> > > > > > great candidates.) > > > >> > > > > > > > > >> > > > > > Looking forward to seeing you all at the event ;) > > > >> > > > > > > > > >> > > > > > Feng > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > > --=20 Austin Weaver Software Engineer FLYR, Inc. www.flyrlabs.com --00000000000053f15d058a876f17--