airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ash Berlin-Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-1557) backfill ignores configured number of slots in a pool
Date Fri, 01 Sep 2017 10:46:03 GMT
Ash Berlin-Taylor created AIRFLOW-1557:
------------------------------------------

             Summary: backfill ignores configured number of slots in a pool
                 Key: AIRFLOW-1557
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1557
             Project: Apache Airflow
          Issue Type: Bug
    Affects Versions: 1.8.1
            Reporter: Ash Berlin-Taylor
         Attachments: Screen Shot 2017-09-01 at 11.39.32.png

I ran a backfill command with this tool:
The backfill process appears to run as many tasks as possible, even when the pool it is running
in should limit the numbers:

{noformat}
airflow backfill \
  -t fetch_dk_unfiltered \
  --pool brand_index_api \
  -s 2017-07-31 -e 2017-08-31 \
  -x \
  brand_index_fetcher 
{noformat}

(Nothing else than the backfill is currently using this pool. I wasn't able to capture a screen
shot of the task instances before the jobs completed.)




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message