airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Scott Halgrim <scott.halg...@zapier.com.INVALID>
Subject Re: Airflow Deployment tools
Date Tue, 14 Nov 2017 17:13:44 GMT
We have a bash script. Main features


• Uses AWS metadata URL to determine if it’s a prod box or dev box and pulls appropriate
branches
• Gets difference between commit on box and commit on remote tip and git logs the difference
into Slack
• Restarts all the upstart bits


That’s on the worker. On the scheduler we just do a git pull and manual restart of upstart
bits :-P

So yeah, this needs to be done with better tooling and I’d love to hear what others are
using, too

Scott


On Nov 14, 2017, 9:02 AM -0800, Andy Hadjigeorgiou <andyxhadji@gmail.com>, wrote:
> Hey,
>
> Was just wondering what tools & services everyone uses to deploy new
> versions of their data pipelines (understandably this would vary greatly
> based on tech stack) but I'd love to hear what the community has been using.
>
> - Andy

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message