airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Crawford <michael.crawf...@modernizingmedicine.com>
Subject Re: Airflow talk @ Coolblue
Date Fri, 03 Nov 2017 14:32:16 GMT
Any chance your talk was recorded?

Thanks,
Mike


> On Oct 29, 2017, at 6:29 AM, Gerard Toonstra <gtoonstra@gmail.com> wrote:
> 
> Hi all,
> 
> Thursday the 26/10 my employer Coolblue organized a "Behind the Scenes"
> event. It is an opportunity for engineers to talk about stuff they work on
> and usually they provide two presentations.
> 
> This event was about BigData and Processing. As (now) team lead of Data
> Platform, I decided to talk about Apache Airflow, which we are now in the
> process of migrating to (from Azkaban).
> 
> Here are the slides:
> 
> https://www.linkedin.com/feed/update/urn:li:activity:6330346647347875840
> 
> It is a technical presentation, aimed at informing people who are new to
> Airflow what the underlying architecture is and also presenting the why
> you'd want to use it in the first place. I based the architectural diagrams
> on AWS on the PoC we did some time.
> 
> Important takeaway:
> 
> Airflow is built around some great design principles, which are the result
> of important insights into data processing. These principles result in a
> tool, when used correctly according to these principles, to reduce the ETL
> effort and maintenance and make time to work on higher level intelligent
> work like Machine Learning, Deep Learning and analysis of your data.
> 
> It is very similar to the talk I gave at BigData Week London 2017:
> 
> https://youtu.be/Ch2AQhOhefw
> 
> Rgds,
> 
> Gerard


Mime
View raw message