airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Egor Mateshuk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AIRFLOW-1557) backfill ignores configured number of slots in a pool
Date Wed, 07 Mar 2018 13:38:00 GMT

    [ https://issues.apache.org/jira/browse/AIRFLOW-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389558#comment-16389558
] 

Egor Mateshuk commented on AIRFLOW-1557:
----------------------------------------

Confirm it's still there

Airflow versionĀ 1.8.2

!Backfill_bug_1.8.2.jpg!

> backfill ignores configured number of slots in a pool
> -----------------------------------------------------
>
>                 Key: AIRFLOW-1557
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1557
>             Project: Apache Airflow
>          Issue Type: Bug
>    Affects Versions: 1.8.1
>            Reporter: Ash Berlin-Taylor
>            Priority: Major
>         Attachments: Backfill_bug_1.8.2.jpg, Screen Shot 2017-09-01 at 11.39.32.png
>
>
> I ran a backfill command with this tool:
> The backfill process appears to run as many tasks as possible, even when the pool it
is running in should limit the numbers:
> {noformat}
> airflow backfill \
>   -t fetch_dk_unfiltered \
>   --pool brand_index_api \
>   -s 2017-07-31 -e 2017-08-31 \
>   -x \
>   brand_index_fetcher 
> {noformat}
> (Nothing else than the backfill is currently using this pool. I wasn't able to capture
a screen shot of the task instances before the jobs completed.)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message