airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Bollinger <m...@lumoslabs.com>
Subject Re: Airflow Deployment tools
Date Wed, 15 Nov 2017 02:19:19 GMT
Samson <https://github.com/zendesk/samson> deploy that runs a script
running a Broadside <https://github.com/lumoslabs/broadside> deploy for
ECS, which bounces the Web and Scheduler workers, and updates the DAG
directory on the workers. Docker images come from a Github -> Travis -> Quay
<https://quay.io/> CI setup.

On Tue, Nov 14, 2017 at 10:18 AM, Alek Storm <alek.storm@gmail.com> wrote:

> Our TeamCity server detects the master branch has changed, then packages up
> the repo containing our DAGs as an artifact. We then use SaltStack to
> trigger a bash script on the targeted servers that downloads the artifact,
> moves the files to the right place, and restarts the scheduler (on the
> master).
>
> This allows us to easily revert changes by redeploying a particular
> TeamCity artifact, without touching the git history.
>
> Alek
>
> On Nov 14, 2017 11:02 AM, "Andy Hadjigeorgiou" <andyxhadji@gmail.com>
> wrote:
>
> > Hey,
> >
> > Was just wondering what tools & services everyone uses to deploy new
> > versions of their data pipelines (understandably this would vary greatly
> > based on tech stack) but I'd love to hear what the community has been
> > using.
> >
> > - Andy
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message