airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Halgrim <>
Subject Re: Airflow Deployment tools
Date Tue, 14 Nov 2017 17:13:44 GMT
We have a bash script. Main features

• Uses AWS metadata URL to determine if it’s a prod box or dev box and pulls appropriate
• Gets difference between commit on box and commit on remote tip and git logs the difference
into Slack
• Restarts all the upstart bits

That’s on the worker. On the scheduler we just do a git pull and manual restart of upstart
bits :-P

So yeah, this needs to be done with better tooling and I’d love to hear what others are
using, too


On Nov 14, 2017, 9:02 AM -0800, Andy Hadjigeorgiou <>, wrote:
> Hey,
> Was just wondering what tools & services everyone uses to deploy new
> versions of their data pipelines (understandably this would vary greatly
> based on tech stack) but I'd love to hear what the community has been using.
> - Andy

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message