airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joy Gao <j...@wepay.com>
Subject Re: Q1 Airflow Bay Area Meetup
Date Thu, 15 Feb 2018 18:20:20 GMT
Hi Sumit!

The date has been finalized. That said, I will attempt to live-stream or
record the event for folks who cannot attend in person. Appreciate your
understanding.

Thanks,
Joy

On Thu, Feb 15, 2018 at 12:55 AM, Sumit Maheshwari <msumit@apache.org>
wrote:

> We can present our internal tool for doing data quality checks for ETL's
>> in Qubole.
>> Iris is an Airflow Operator that currently has built-in functions for count
>> validation, data-profile validation and duplicate data validation.
>
>
> That would be great Devjyoti.
>
>
> folks,
>
> I would really love to attend this meetup and meet the Airflow community.
> I'll be attending dataplatforms  <https://www.dataplatforms.com/>conference
> between 11-13th Apr and would be in the bay area before and after that.
>
> On the Airflow meetup page, I see that it's scheduled on 11th Apr, so if
> possible can we move it to 13th or some other day, it would be great.
>
>
> Thanks,
> Sumit
>
>
> On Wed, Jan 31, 2018 at 1:52 PM, Devjyoti Patra <devjyotip@qubole.com>
> wrote:
>
>> We can present our internal tool for doing data quality checks for ETL's
>> in
>> Qubole.
>> Iris is an Airflow Operator that currently has built-in functions for
>>  count validation,
>> data-profile validation and duplicate data validation.
>>
>> On Wed, Jan 31, 2018 at 3:10 AM, George Leslie-Waksman <
>> george@cloverhealth.com.invalid> wrote:
>>
>> > If you're still looking for speakers, I have a handful of things I could
>> > pull together to talk about:
>> >
>> >    - the interplay between pools, queues, parallelism, dag concurrency,
>> and
>> >    other job scheduling levers
>> >    - layering additional instrumentation on top of Airflow operators
>> >    - integrity testing DAGs in CI before they hit production / unit
>> testing
>> >    tasks
>> >    - using Airflow metadata to track systemic execution behavior
>> >    - how I would have used Airflow if I'd known then what I know now
>> >
>> >
>> > On Thu, Jan 11, 2018 at 12:18 PM Andrew Maguire <andrewm4894@gmail.com>
>> > wrote:
>> >
>> > > +1 on being able to watch a recording, based in Dublin, Ireland but
>> also
>> > > very interested in the cloud side of airflow.
>> > >
>> > > On Thu, 11 Jan 2018, 17:59 Joy Gao, <joyg@wepay.com> wrote:
>> > >
>> > > > Great to see that there's a lot of interests!
>> > > >
>> > > > I will go ahead and work on organizing this event. Details TBD.
>> > > >
>> > > > On Wed, Jan 10, 2018 at 8:03 AM, Laura Lorenz <
>> > llorenz@industrydive.com>
>> > > > wrote:
>> > > >
>> > > > > I would love to see videos from this as well, as we're east
>> coast. We
>> > > > host
>> > > > > our airflow installation in GCP using GKE, and would love to
>> compare
>> > > > notes.
>> > > > >
>> > > > > Laura
>> > > > >
>> > > > > On Tue, Jan 9, 2018 at 4:57 AM, Bolke de Bruin <bdbruin@gmail.com
>> >
>> > > > wrote:
>> > > > >
>> > > > > > If it coincides with qcon.ai (april 10-11) or is close to
its
>> > dates
>> > > > > then I
>> > > > > > can join.
>> > > > > >
>> > > > > > B.
>> > > > > >
>> > > > > > Op 9 jan. 2018 5:36 a.m. schreef "Ananth Durai" <
>> > vananth22@gmail.com
>> > > >:
>> > > > > >
>> > > > > > I can give a talk about all the hacks we did to scale Airflow
>> Local
>> > > > > > Executor and improve the data pipeline on-call experience
at
>> Slack
>> > if
>> > > > > folks
>> > > > > > are interested.
>> > > > > >
>> > > > > > Regards,
>> > > > > > Ananth.P,
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On 8 January 2018 at 15:51, George Leslie-Waksman <
>> > > > > > george@cloverhealth.com.invalid> wrote:
>> > > > > >
>> > > > > > > +1 and would love to hear about other folks running
Airflow in
>> > GCP.
>> > > > > > >
>> > > > > > > Would strongly prefer March (but I'm just one person)
>> > > > > > >
>> > > > > > > On Mon, Jan 8, 2018 at 2:30 PM Feng Lu
>> <fenglu@google.com.invalid
>> > >
>> > > > > > wrote:
>> > > > > > >
>> > > > > > > > +1 Joy!
>> > > > > > > >
>> > > > > > > > Would prefer April if possible, I can talk about
running
>> > Airflow
>> > > in
>> > > > > GCP
>> > > > > > > if
>> > > > > > > > there's sufficient interest.
>> > > > > > > >
>> > > > > > > > On Fri, Jan 5, 2018 at 12:59 PM, Sid Anand <
>> sanand@apache.org>
>> > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Sounds great Joy!
>> > > > > > > > >
>> > > > > > > > > I've promoted you to *Event Organizer* on
the Bay Area
>> Apache
>> > > > > Airflow
>> > > > > > > > > Meetup as per instructions/guidelines on
>> > > > > > > > > https://cwiki.apache.org/confl
>> uence/display/AIRFLOW/Meetups
>> > > > > > > > >
>> > > > > > > > > Go ahead and start setting up the meetup...
we recommend 3
>> > > > speakers
>> > > > > > > with
>> > > > > > > > > 1-2 of them being external to the hosting
company.
>> > > > > > > > >
>> > > > > > > > > Once the meetup date is setup, we can add
it to
>> > > > > > > > >
>> > > > https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements
>> > > > > &
>> > > > > > > > tweet
>> > > > > > > > > it or share it over the dev list.
>> > > > > > > > > -s
>> > > > > > > > >
>> > > > > > > > > On Fri, Jan 5, 2018 at 12:10 PM, Joy Gao
<joyg@wepay.com>
>> > > wrote:
>> > > > > > > > >
>> > > > > > > > > > Hi folks,
>> > > > > > > > > >
>> > > > > > > > > > I'm Joy from WePay. At the last Airflow
meetup in
>> December
>> > > > there
>> > > > > > was
>> > > > > > > a
>> > > > > > > > > > demand for hosting a future meetup to
cover topics on
>> > Airflow
>> > > > > > > > > integrations
>> > > > > > > > > > with GCP/AWS (for example, CI/CD with
GCP/AWS
>> > > hooks/operators,
>> > > > > best
>> > > > > > > > > > practices on running Airflow in the
cloud, managed
>> Airflow,
>> > > > etc.)
>> > > > > > > > > >
>> > > > > > > > > > If there is enough interest, WePay can
host the next
>> event
>> > > > > > sometimes
>> > > > > > > in
>> > > > > > > > > > March/April. And if you would like to
give a talk or
>> have
>> > > > another
>> > > > > > > topic
>> > > > > > > > > > covered, feel free to suggest them as
well.
>> > > > > > > > > >
>> > > > > > > > > > Cheers,
>> > > > > > > > > > Joy
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message