airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kaxil Naik <kaxiln...@gmail.com>
Subject Re: Why is Airflow 1.10.12 much slower than 1.10.10?
Date Tue, 24 Nov 2020 12:22:19 GMT
Can you test your DAG with 1.10.13rc1, it contains some performance
improvements that should take care of it.

You can install it as follows:

*pip install apache-airflow==1.10.13rc1*

Regards,
Kaxil

On Tue, Nov 24, 2020 at 3:57 AM lunbingli <lunbingli@163.com> wrote:

> Recently I have my airflow upgraded from 1.10.10 to 1.10.12 and find my
> dags are scheduled much slower than before. One of my dag is as
> follows(please check attachment) and it has 400 Tasks of DummyOperator.*
> It took only 4+mins with 1.10.10 while it took more than 16mins with
> 1.10.12.*
> *Can someone tell me what the problem may be?*I tried to compare the code
> diff between these two versions with
> https://github.com/apache/airflow/compare/1.10.10...1.10.12
> <https://github.com/apache/airflow/discussions/url> and failed to draw a
> conclusion yet.
>
>
>
>

Mime
View raw message