airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Capwell <dcapw...@gmail.com>
Subject Re: Airflow with Celery
Date Tue, 15 May 2018 19:12:10 GMT
What I find is that when celery rejects we hit this.  For us we don't do
work on the hosts so solve by over provisioning tasks in celery

On Tue, May 15, 2018, 6:30 AM Andy Cooper <andy.cooper@astronomer.io> wrote:

> I have had very similar issues when there was a problem with the connection
> string pointing to the message broker. Triple check those connection
> strings and attempt to connect outside of airflow.
>
> On Tue, May 15, 2018 at 9:27 AM Goutham Pratapa <pratapagoutham@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I have been using airflow with Celery executor in the background
> >
> > https://hastebin.com/sipecovomi.ini --> airflow.cfg
> >
> > https://hastebin.com/urutokuvoq.py   --> The dag I have been using
> >
> >
> >
> > This shows that the dag is always in running state.
> >
> >
> >
> >
> > Airflow flower shows nothing in the tasks or in the broker.
> >
> >
> > Did I miss anything can anyone help me in this regard.
> >
> >
> > --
> > Cheers !!!
> > Goutham Pratapa
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message