airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Van Boxel <a...@vanboxel.be>
Subject Re: [VOTE] Release Airflow 1.8.0 based on Airflow 1.8.0rc3
Date Sun, 12 Feb 2017 10:17:42 GMT
Running ok, in staging... @bolke I'm running patch-less. I've switched my
Kubernetes from:

 - each container (webserver/scheduler/worker) had a git-sync'er (getting
the dags from git)
  > this meant that the scheduler had 0 dags at startup, and should have
picked them up later

to

 - single NFS share that shares airflow_home over each container
  > the git sync'er is now a seperate container running before the other
containers

This resolved my mystery DAG crashes.

I'll be updating production to a patchless RC3 today, you get my vote after
that.




On Sun, Feb 12, 2017 at 4:59 AM Boris Tyukin <boris@boristyukin.com> wrote:

> awesome! thanks Jeremiah
>
> On Sat, Feb 11, 2017 at 12:53 PM, Jeremiah Lowin <jlowin@apache.org>
> wrote:
>
> > Boris, I submitted a PR to address your second point --
> > https://github.com/apache/incubator-airflow/pull/2068. Thanks!
> >
> > On Sat, Feb 11, 2017 at 10:42 AM Boris Tyukin <boris@boristyukin.com>
> > wrote:
> >
> > > I am running LocalExecutor and not doing crazy things but use DAG
> > > generation heavily - everything runs fine as before. As I mentioned in
> > > other threads only had a few issues:
> > >
> > > 1) had to upgrade MySQL which was a PAIN. Cloudera CDH is running old
> > > version of MySQL which was compatible with 1.7.1 but not compatible now
> > > with 1.8 because of fractional seconds support PR.
> > >
> > > 2) when you install airflow, there are two new example DAGs
> > > (last_task_only) which are going back very far in the past and
> scheduled
> > to
> > > run every hour - a bunch of dags triggered on the first start of
> > scheduler
> > > and hosed my CPU
> > >
> > > Everything else was fine and I LOVE lots of small UI changes, which
> > reduced
> > > a lot my use of cli.
> > >
> > > Thanks again for the amazing work and an awesome project!
> > >
> > >
> > > On Sat, Feb 11, 2017 at 9:17 AM, Jeremiah Lowin <jlowin@apache.org>
> > wrote:
> > >
> > > > I was able to deploy successfully. +1 (binding)
> > > >
> > > > On Fri, Feb 10, 2017 at 7:37 PM Maxime Beauchemin <
> > > > maximebeauchemin@gmail.com> wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > On Fri, Feb 10, 2017 at 3:44 PM, Arthur Wiedmer <
> > > > arthur.wiedmer@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > On Feb 10, 2017 3:13 PM, "Dan Davydov" <dan.davydov@airbnb.com.
> > > > invalid>
> > > > > > wrote:
> > > > > >
> > > > > > > Our staging looks good, all the DAGs there pass.
> > > > > > > +1 (binding)
> > > > > > >
> > > > > > > On Fri, Feb 10, 2017 at 10:21 AM, Chris Riccomini <
> > > > > criccomini@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Running in all environments. Will vote after the weekend
to
> > make
> > > > sure
> > > > > > > > things are working properly, but so far so good.
> > > > > > > >
> > > > > > > > On Fri, Feb 10, 2017 at 6:05 AM, Bolke de Bruin <
> > > bdbruin@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Dear All,
> > > > > > > > >
> > > > > > > > > Let’s try again!
> > > > > > > > >
> > > > > > > > > I have made the THIRD RELEASE CANDIDATE of Airflow
1.8.0
> > > > available
> > > > > > at:
> > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow/
> <
> > > > > > > > > https://dist.apache.org/repos/dist/dev/incubator/airflow/>
> ,
> > > > > public
> > > > > > > keys
> > > > > > > > > are available at https://dist.apache.org/repos/
> > > > > > dist/release/incubator/
> > > > > > > > > airflow/ <
> > > https://dist.apache.org/repos/dist/release/incubator/
> > > > > > > airflow/>
> > > > > > > > > . It is tagged with a local version “apache.incubating”
so
> it
> > > > > allows
> > > > > > > > > upgrading from earlier releases.
> > > > > > > > >
> > > > > > > > > Two issues have been fixed since release candidate
2:
> > > > > > > > >
> > > > > > > > > * trigger_dag could create dags with fractional
seconds,
> not
> > > > > > supported
> > > > > > > by
> > > > > > > > > logging and UI at the moment
> > > > > > > > > * local api client trigger_dag had hardcoded
execution of
> > None
> > > > > > > > >
> > > > > > > > > Known issue:
> > > > > > > > > * Airflow on kubernetes and num_runs -1 (default)
can
> expose
> > > > import
> > > > > > > > issues.
> > > > > > > > >
> > > > > > > > > I have extensively discussed this with Alex (reporter)
and
> we
> > > > > > consider
> > > > > > > > > this a known issue with a workaround available
as we are
> > unable
> > > > to
> > > > > > > > > replicate this in a different environment. UPDATING.md
has
> > been
> > > > > > updated
> > > > > > > > > with the work around.
> > > > > > > > >
> > > > > > > > > As these issues are confined to a very specific
area and
> full
> > > > unit
> > > > > > > tests
> > > > > > > > > were added I would also like to raise a VOTE
for releasing
> > > 1.8.0
> > > > > > based
> > > > > > > on
> > > > > > > > > release candidate 3, i.e. just renaming release
candidate 3
> > to
> > > > > 1.8.0
> > > > > > > > > release.
> > > > > > > > >
> > > > > > > > > Please respond to this email by:
> > > > > > > > >
> > > > > > > > > +1,0,-1 with *binding* if you are a PMC member
or
> > *non-binding*
> > > > if
> > > > > > you
> > > > > > > > are
> > > > > > > > > not.
> > > > > > > > >
> > > > > > > > > Thanks!
> > > > > > > > > Bolke
> > > > > > > > >
> > > > > > > > > My VOTE: +1 (binding)
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
-- 
  _/
_/ Alex Van Boxel

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message