airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Feng Lu <fen...@google.com.INVALID>
Subject Re: Bay Area Airflow meetup
Date Thu, 23 May 2019 06:31:04 GMT
We have now finalized the meetup schedule and logistics, lots of
interesting talks, please check out the details
<https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/260712102/>
here.
Also a big thank you to all the presenters, see you soon!

Feng

On Fri, May 10, 2019 at 3:54 PM Feng Lu <fenglu@google.com> wrote:

> Great, thank you Gabriel.
>
> *From: *Gabriel Silk <gsilk@dropbox.com.invalid>
> *Date: *Fri, May 10, 2019 at 1:44 PM
> *To: * <dev@airflow.apache.org>
> *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes
>
> Sure, how about:
>>
>> "Migrating Dropbox to Airflow, and the challenges of ETL at scale"
>>
>> Abstract: Dropbox has over 20k batch jobs that run on an hourly and/or
>> daily cadence, connected together in a large web of dependencies.
>> Migrating
>> this ETL to Airflow has proven to be an interesting challenge. In this
>> talk
>> we'll discover how we went about it, what challenges we are facing, how
>> Airflow is helping us, and opportunities we see ahead"
>>
>> On Fri, May 10, 2019 at 1:05 PM Feng Lu <fenglu@google.com.invalid>
>> wrote:
>>
>> > Hi Gabriel,
>> >
>> > Awesome, could you share with us the talk title and abstract?
>> > Thanks.
>> >
>> > Feng
>> >
>> > *From: *Gabriel Silk <gsilk@dropbox.com.invalid>
>> > *Date: *Fri, May 10, 2019 at 10:15 AM
>> > *To: * <dev@airflow.apache.org>
>> > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes
>> >
>> > Hey Feng,
>> > >
>> > > I'm the tech lead on Data Infrastructure at Dropbox, and I'd like to
>> talk
>> > > about the challenges and opportunities in migrating large-scale ETL to
>> > > Airflow from a system that was built in-house.
>> > >
>> > >
>> > > On Fri, May 10, 2019 at 8:23 AM Szymon Przedwojski <
>> > > szymon.przedwojski@polidea.com> wrote:
>> > >
>> > > > Hi Feng,
>> > > >
>> > > > I'd like to submit my talk proposal for the Bay Area meetup.
>> > > >
>> > > > *Working title:* Migrating Apache Oozie Workflows to Apache Airflow
>> -
>> > the
>> > > > developer’s perspective
>> > > >
>> > > > I want to talk about the Oozie-to-Airflow [1] project created by
>> Google
>> > > > that we at Polidea help develop.
>> > > > It's a conversion tool written in Python that generates Airflow
>> Python
>> > > DAGs
>> > > > from Oozie workflow XMLs, thus enabling easy migration of Oozie
>> > workflows
>> > > > to Airflow.
>> > > > I would like to present the architecture of the solution and of the
>> > whole
>> > > > end-to-end system tests suite, talk about the challenges we faced
>> when
>> > > > translating various Oozie features to Airflow and discuss the
>> future of
>> > > the
>> > > > project.
>> > > > Moreover, I will show how the Airflow community can get involved in
>> the
>> > > > open-source Oozie-to-Airflow project and why it would be mutually
>> > > > beneficial.
>> > > >
>> > > > *About me:*
>> > > > I’m a Software Engineer @ Polidea in Warsaw. I’ve been programming
>> > > > professionally for 5 years now, mainly using Python, Java and JS.
>> > > > I have public speaking experience - I gave a couple of talks at
>> Python
>> > > and
>> > > > JS meetups, at a GDG Dev Fest Warsaw event and multiple internal
>> talks
>> > at
>> > > > companies I’ve worked for.
>> > > > I’ve been involved with Airflow since August last year and I’ve
>> > > contributed
>> > > > several operators for different GCP services.
>> > > >
>> > > > [1] https://github.com/GoogleCloudPlatform/cloud-composer
>> > > >
>> > > > *Szymon Przedwojski*
>> > > > Polidea | Software Engineer
>> > > >
>> > > > M: +48 500 330 790
>> > > >
>> > > >
>> > > > On Thu, 18 Apr 2019 at 01:08, Feng Lu <fenglu@google.com.invalid>
>> > wrote:
>> > > >
>> > > > > We've finalized the meetup logistics (details here
>> > > > > <
>> > > > >
>> > > >
>> > >
>> >
>> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/260712102/
>> > > > > >).
>> > > > > Please send in your talk proposals!
>> > > > > Thank you.
>> > > > >
>> > > > > Feng
>> > > > >
>> > > > > On Tue, Mar 12, 2019 at 1:15 PM Feng Lu <fenglu@google.com>
>> wrote:
>> > > > >
>> > > > > > Hi all,
>> > > > > >
>> > > > > > It has been a while since we had the last Bay Area meetup
and we
>> > > > (Google)
>> > > > > > are going to host another meetup around June this year.
Will
>> share
>> > > with
>> > > > > > everyone the logistic details, meanwhile, please feel free
to
>> send
>> > in
>> > > > > your
>> > > > > > talk proposals!
>> > > > > > (I see a lot of very interesting AIP ideas floating around,
>> these
>> > are
>> > > > > > great candidates.)
>> > > > > >
>> > > > > > Looking forward to seeing you all at the event ;)
>> > > > > >
>> > > > > > Feng
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message