airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Maheshwari <msu...@apache.org>
Subject Re: Q1 Airflow Bay Area Meetup
Date Thu, 15 Feb 2018 08:55:51 GMT
>
> We can present our internal tool for doing data quality checks for ETL's
> in Qubole.
> Iris is an Airflow Operator that currently has built-in functions for count
> validation, data-profile validation and duplicate data validation.


That would be great Devjyoti.


folks,

I would really love to attend this meetup and meet the Airflow community.
I'll be attending dataplatforms  <https://www.dataplatforms.com/>conference
between 11-13th Apr and would be in the bay area before and after that.

On the Airflow meetup page, I see that it's scheduled on 11th Apr, so if
possible can we move it to 13th or some other day, it would be great.


Thanks,
Sumit


On Wed, Jan 31, 2018 at 1:52 PM, Devjyoti Patra <devjyotip@qubole.com>
wrote:

> We can present our internal tool for doing data quality checks for ETL's in
> Qubole.
> Iris is an Airflow Operator that currently has built-in functions for
>  count validation,
> data-profile validation and duplicate data validation.
>
> On Wed, Jan 31, 2018 at 3:10 AM, George Leslie-Waksman <
> george@cloverhealth.com.invalid> wrote:
>
> > If you're still looking for speakers, I have a handful of things I could
> > pull together to talk about:
> >
> >    - the interplay between pools, queues, parallelism, dag concurrency,
> and
> >    other job scheduling levers
> >    - layering additional instrumentation on top of Airflow operators
> >    - integrity testing DAGs in CI before they hit production / unit
> testing
> >    tasks
> >    - using Airflow metadata to track systemic execution behavior
> >    - how I would have used Airflow if I'd known then what I know now
> >
> >
> > On Thu, Jan 11, 2018 at 12:18 PM Andrew Maguire <andrewm4894@gmail.com>
> > wrote:
> >
> > > +1 on being able to watch a recording, based in Dublin, Ireland but
> also
> > > very interested in the cloud side of airflow.
> > >
> > > On Thu, 11 Jan 2018, 17:59 Joy Gao, <joyg@wepay.com> wrote:
> > >
> > > > Great to see that there's a lot of interests!
> > > >
> > > > I will go ahead and work on organizing this event. Details TBD.
> > > >
> > > > On Wed, Jan 10, 2018 at 8:03 AM, Laura Lorenz <
> > llorenz@industrydive.com>
> > > > wrote:
> > > >
> > > > > I would love to see videos from this as well, as we're east coast.
> We
> > > > host
> > > > > our airflow installation in GCP using GKE, and would love to
> compare
> > > > notes.
> > > > >
> > > > > Laura
> > > > >
> > > > > On Tue, Jan 9, 2018 at 4:57 AM, Bolke de Bruin <bdbruin@gmail.com>
> > > > wrote:
> > > > >
> > > > > > If it coincides with qcon.ai (april 10-11) or is close to its
> > dates
> > > > > then I
> > > > > > can join.
> > > > > >
> > > > > > B.
> > > > > >
> > > > > > Op 9 jan. 2018 5:36 a.m. schreef "Ananth Durai" <
> > vananth22@gmail.com
> > > >:
> > > > > >
> > > > > > I can give a talk about all the hacks we did to scale Airflow
> Local
> > > > > > Executor and improve the data pipeline on-call experience at
> Slack
> > if
> > > > > folks
> > > > > > are interested.
> > > > > >
> > > > > > Regards,
> > > > > > Ananth.P,
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 8 January 2018 at 15:51, George Leslie-Waksman <
> > > > > > george@cloverhealth.com.invalid> wrote:
> > > > > >
> > > > > > > +1 and would love to hear about other folks running Airflow
in
> > GCP.
> > > > > > >
> > > > > > > Would strongly prefer March (but I'm just one person)
> > > > > > >
> > > > > > > On Mon, Jan 8, 2018 at 2:30 PM Feng Lu
> <fenglu@google.com.invalid
> > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > +1 Joy!
> > > > > > > >
> > > > > > > > Would prefer April if possible, I can talk about running
> > Airflow
> > > in
> > > > > GCP
> > > > > > > if
> > > > > > > > there's sufficient interest.
> > > > > > > >
> > > > > > > > On Fri, Jan 5, 2018 at 12:59 PM, Sid Anand <
> sanand@apache.org>
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Sounds great Joy!
> > > > > > > > >
> > > > > > > > > I've promoted you to *Event Organizer* on the
Bay Area
> Apache
> > > > > Airflow
> > > > > > > > > Meetup as per instructions/guidelines on
> > > > > > > > > https://cwiki.apache.org/confluence/display/AIRFLOW/
> Meetups
> > > > > > > > >
> > > > > > > > > Go ahead and start setting up the meetup... we
recommend 3
> > > > speakers
> > > > > > > with
> > > > > > > > > 1-2 of them being external to the hosting company.
> > > > > > > > >
> > > > > > > > > Once the meetup date is setup, we can add it
to
> > > > > > > > >
> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements
> > > > > &
> > > > > > > > tweet
> > > > > > > > > it or share it over the dev list.
> > > > > > > > > -s
> > > > > > > > >
> > > > > > > > > On Fri, Jan 5, 2018 at 12:10 PM, Joy Gao <joyg@wepay.com>
> > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi folks,
> > > > > > > > > >
> > > > > > > > > > I'm Joy from WePay. At the last Airflow
meetup in
> December
> > > > there
> > > > > > was
> > > > > > > a
> > > > > > > > > > demand for hosting a future meetup to cover
topics on
> > Airflow
> > > > > > > > > integrations
> > > > > > > > > > with GCP/AWS (for example, CI/CD with GCP/AWS
> > > hooks/operators,
> > > > > best
> > > > > > > > > > practices on running Airflow in the cloud,
managed
> Airflow,
> > > > etc.)
> > > > > > > > > >
> > > > > > > > > > If there is enough interest, WePay can host
the next
> event
> > > > > > sometimes
> > > > > > > in
> > > > > > > > > > March/April. And if you would like to give
a talk or have
> > > > another
> > > > > > > topic
> > > > > > > > > > covered, feel free to suggest them as well.
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > > Joy
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message