airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <lance.nors...@gmail.com>
Subject Re: About Pools and Task Retries
Date Sat, 30 Jul 2016 23:05:05 GMT
The task copies a table from the application DB to Redshift for analytics
work, once a day. If the application DB is super-busy, I want the task to
back off for many minutes and retry later.

So, if I want a timeout of 1/2 an hour, I should have the error callback
leave the pool and the retry callback enter the pool again?

On Sat, Jul 30, 2016 at 3:31 AM, Bolke de Bruin <bdbruin@gmail.com> wrote:

> Yes it does.
>
> (Tasks are not really aware of pools / slots, I have some work in that
> direction but that will be ready after August)
>
> Sent from my iPhone
>
> > On 30 jul. 2016, at 01:35, Lance Norskog <lance.norskog@gmail.com>
> wrote:
> >
> > When a Task in in a Pool, fails, and goes to sleep to retry, does it give
> > up its Pool membership while asleep?
> >
> >
> > --
> > Lance Norskog
> > lance.norskog@gmail.com
> > Redwood City, CA
>



-- 
Lance Norskog
lance.norskog@gmail.com
Redwood City, CA

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message