airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Maguire <andrewm4...@gmail.com>
Subject Re: Runbook to upgrade Airflow
Date Mon, 16 Oct 2017 14:48:14 GMT
would love to have a sort of step by step here.

upgrading to 1.9 now as want to help as best i can and figure installing
latest version one good way to help find any issues.

unless its as simple as pip install.

So far steps i'm taking (we just run local on single server as use BigQuery
for most work)

1. wget
https://dist.apache.org/repos/dist/dev/incubator/airflow/1.9.0alpha1/apache-airflow-1.9.0alpha1+incubating-bin.tar.gz
2. pip install apache-airflow-1.9.0alpha1+incubating-bin.tar.gz
3. reboot server
4. airflow upgradedb

Anything i might be missing here or anything dangerous in what i'm doing?

Cheers,
Andy

On Wed, Oct 4, 2017 at 10:09 PM Thoralf Gutierrez <
thoralfgutierrez@gmail.com> wrote:

> I see - thanks for the answers!
>
> On Wed, 4 Oct 2017 at 13:10 Alex Guziel <alex.guziel@airbnb.com.invalid>
> wrote:
>
> > We pretty much just shut everything off (fence the db by changing the
> > password), upgrade the db, then turn everything back on and change the db
> > password back. We accept that there will be downtime for maintenance and
> do
> > it at a low impact time (like 20:00 utc ). Also, the thing with
> sqlalchemy
> > is a lot more things become breaking changes because of the way it writes
> > queries.
> >
> > It might be good to have downtime even without schema changes to prevent
> > weird behavior if the changeset is large.
> >
> > But for small stuff, depending on the scope of the change, workers don't
> > really need to be restarted, scheduler can be handled manually, and
> > webserver we do a rolling restart.
> >
> > On Wed, Oct 4, 2017 at 7:54 AM Thoralf Gutierrez <
> > thoralfgutierrez@gmail.com>
> > wrote:
> >
> > > Thanks for your answer Alex.
> > >
> > > I guess you mean it won't work if there is a _breaking_ schema change
> > > right? But for new patch and minor versions (i.e. without breaking
> > > changes), I'm guessing that upgrading the db first should do the trick?
> > >
> > > @Airbnb team, how do you all do it? If I remember correctly, you often
> > > update your version of airflow to test out new releases. What does your
> > > runbook for Airflow upgrades look like?
> > >
> > > Cheers,
> > > Thoralf
> > >
> > > On Tue, 3 Oct 2017 at 13:04 Alex Guziel <alex.guziel@airbnb.com
> .invalid>
> > > wrote:
> > >
> > > > You won't be able to if there's a schema change.
> > > >
> > > > On Tue, Oct 3, 2017 at 12:33 PM, Thoralf Gutierrez <
> > > > thoralfgutierrez@gmail.com> wrote:
> > > >
> > > > > Hey everybody!
> > > > >
> > > > > Does anybody have some kind of runbook to upgrade airflow (with a
> > > Celery
> > > > > backend) without having any downtime (i.e. tasks keep on running
as
> > you
> > > > > upgrade)?
> > > > >
> > > > > I have this in mind, but not sure if I am missing something or if
I
> > > > should
> > > > > be careful with the order of steps (especially for upgrading the
db
> > > > > schema):
> > > > >
> > > > > 1. run airflow upgradedb from anywhere
> > > > >
> > > > > 2. one worker at a time
> > > > >   2a. make sure it doesn't start any new task.
> > > > >   2b. wait for all tasks to be finished
> > > > >   2c. run pip install airflow --upgrade
> > > > >   2d. re-enable worker
> > > > >
> > > > > 3. one webserver at a time
> > > > >   3a. kill webserver
> > > > >   3b. run pip install airflow --upgrade
> > > > >   3c. start webserver
> > > > >
> > > > > 4. scheduler
> > > > >   4a. kill scheduler
> > > > >   4b. run pip intsall airflow --upgrade
> > > > >   4c. start scheduler
> > > > >
> > > > > Thanks,
> > > > > Thoralf
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message