airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Austin Weaver <aus...@flyrlabs.com>
Subject Re: Bay Area Airflow meetup
Date Tue, 04 Jun 2019 22:59:25 GMT
Thanks for the reminder! Unfortunately the same time as the warriors are
playing the nba finals in oakland ;(

Is there any chance of this being recorded?

Thanks!
Austin

On Tue, Jun 4, 2019 at 3:40 PM Feng Lu <fenglu@google.com.invalid> wrote:

> A gentle reminder, the meetup will be tomorrow and we just updated the
> event
> location
> <
> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/260712102/
> >
> to accommodate more people.
> See you there.
>
> On Thu, May 23, 2019 at 11:08 AM Aizhamal Nurmamat kyzy
> <aizhamal@google.com.invalid> wrote:
>
> > Great list of speakers and topics! Will there be a live-streaming?
> >
> > Looking forward to the talks :)
> >
> > On Wed, May 22, 2019 at 11:48 PM Feng Lu <fenglu@google.com.invalid>
> > wrote:
> >
> > > We have now finalized the meetup schedule and logistics, lots of
> > > interesting talks, please check out the details
> > > <
> > >
> >
> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/260712102/
> > > >
> > > here.
> > > Also a big thank you to all the presenters, see you soon!
> > >
> > > Feng
> > >
> > > On Fri, May 10, 2019 at 3:54 PM Feng Lu <fenglu@google.com> wrote:
> > >
> > > > Great, thank you Gabriel.
> > > >
> > > > *From: *Gabriel Silk <gsilk@dropbox.com.invalid>
> > > > *Date: *Fri, May 10, 2019 at 1:44 PM
> > > > *To: * <dev@airflow.apache.org>
> > > > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes
> > > >
> > > > Sure, how about:
> > > >>
> > > >> "Migrating Dropbox to Airflow, and the challenges of ETL at scale"
> > > >>
> > > >> Abstract: Dropbox has over 20k batch jobs that run on an hourly
> and/or
> > > >> daily cadence, connected together in a large web of dependencies.
> > > >> Migrating
> > > >> this ETL to Airflow has proven to be an interesting challenge. In
> this
> > > >> talk
> > > >> we'll discover how we went about it, what challenges we are facing,
> > how
> > > >> Airflow is helping us, and opportunities we see ahead"
> > > >>
> > > >> On Fri, May 10, 2019 at 1:05 PM Feng Lu <fenglu@google.com.invalid>
> > > >> wrote:
> > > >>
> > > >> > Hi Gabriel,
> > > >> >
> > > >> > Awesome, could you share with us the talk title and abstract?
> > > >> > Thanks.
> > > >> >
> > > >> > Feng
> > > >> >
> > > >> > *From: *Gabriel Silk <gsilk@dropbox.com.invalid>
> > > >> > *Date: *Fri, May 10, 2019 at 10:15 AM
> > > >> > *To: * <dev@airflow.apache.org>
> > > >> > *Cc: *Griselda Cuevas, Leah Cole, Trishka Fernandes
> > > >> >
> > > >> > Hey Feng,
> > > >> > >
> > > >> > > I'm the tech lead on Data Infrastructure at Dropbox, and
I'd
> like
> > to
> > > >> talk
> > > >> > > about the challenges and opportunities in migrating large-scale
> > ETL
> > > to
> > > >> > > Airflow from a system that was built in-house.
> > > >> > >
> > > >> > >
> > > >> > > On Fri, May 10, 2019 at 8:23 AM Szymon Przedwojski <
> > > >> > > szymon.przedwojski@polidea.com> wrote:
> > > >> > >
> > > >> > > > Hi Feng,
> > > >> > > >
> > > >> > > > I'd like to submit my talk proposal for the Bay Area
meetup.
> > > >> > > >
> > > >> > > > *Working title:* Migrating Apache Oozie Workflows to
Apache
> > > Airflow
> > > >> -
> > > >> > the
> > > >> > > > developer’s perspective
> > > >> > > >
> > > >> > > > I want to talk about the Oozie-to-Airflow [1] project
created
> by
> > > >> Google
> > > >> > > > that we at Polidea help develop.
> > > >> > > > It's a conversion tool written in Python that generates
> Airflow
> > > >> Python
> > > >> > > DAGs
> > > >> > > > from Oozie workflow XMLs, thus enabling easy migration
of
> Oozie
> > > >> > workflows
> > > >> > > > to Airflow.
> > > >> > > > I would like to present the architecture of the solution
and
> of
> > > the
> > > >> > whole
> > > >> > > > end-to-end system tests suite, talk about the challenges
we
> > faced
> > > >> when
> > > >> > > > translating various Oozie features to Airflow and discuss
the
> > > >> future of
> > > >> > > the
> > > >> > > > project.
> > > >> > > > Moreover, I will show how the Airflow community can
get
> involved
> > > in
> > > >> the
> > > >> > > > open-source Oozie-to-Airflow project and why it would
be
> > mutually
> > > >> > > > beneficial.
> > > >> > > >
> > > >> > > > *About me:*
> > > >> > > > I’m a Software Engineer @ Polidea in Warsaw. I’ve
been
> > programming
> > > >> > > > professionally for 5 years now, mainly using Python,
Java and
> > JS.
> > > >> > > > I have public speaking experience - I gave a couple
of talks
> at
> > > >> Python
> > > >> > > and
> > > >> > > > JS meetups, at a GDG Dev Fest Warsaw event and multiple
> internal
> > > >> talks
> > > >> > at
> > > >> > > > companies I’ve worked for.
> > > >> > > > I’ve been involved with Airflow since August last
year and
> I’ve
> > > >> > > contributed
> > > >> > > > several operators for different GCP services.
> > > >> > > >
> > > >> > > > [1] https://github.com/GoogleCloudPlatform/cloud-composer
> > > >> > > >
> > > >> > > > *Szymon Przedwojski*
> > > >> > > > Polidea | Software Engineer
> > > >> > > >
> > > >> > > > M: +48 500 330 790
> > > >> > > >
> > > >> > > >
> > > >> > > > On Thu, 18 Apr 2019 at 01:08, Feng Lu
> <fenglu@google.com.invalid
> > >
> > > >> > wrote:
> > > >> > > >
> > > >> > > > > We've finalized the meetup logistics (details
here
> > > >> > > > > <
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://www.meetup.com/Bay-Area-Apache-Airflow-Incubating-Meetup/events/260712102/
> > > >> > > > > >).
> > > >> > > > > Please send in your talk proposals!
> > > >> > > > > Thank you.
> > > >> > > > >
> > > >> > > > > Feng
> > > >> > > > >
> > > >> > > > > On Tue, Mar 12, 2019 at 1:15 PM Feng Lu <fenglu@google.com>
> > > >> wrote:
> > > >> > > > >
> > > >> > > > > > Hi all,
> > > >> > > > > >
> > > >> > > > > > It has been a while since we had the last
Bay Area meetup
> > and
> > > we
> > > >> > > > (Google)
> > > >> > > > > > are going to host another meetup around June
this year.
> Will
> > > >> share
> > > >> > > with
> > > >> > > > > > everyone the logistic details, meanwhile,
please feel free
> > to
> > > >> send
> > > >> > in
> > > >> > > > > your
> > > >> > > > > > talk proposals!
> > > >> > > > > > (I see a lot of very interesting AIP ideas
floating
> around,
> > > >> these
> > > >> > are
> > > >> > > > > > great candidates.)
> > > >> > > > > >
> > > >> > > > > > Looking forward to seeing you all at the
event ;)
> > > >> > > > > >
> > > >> > > > > > Feng
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > >
> >
>


-- 
Austin Weaver
Software Engineer
FLYR, Inc.   www.flyrlabs.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message