airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anandgupta1412@gmail.com <anandgupta1...@gmail.com>
Subject Large number of SQL Connection from airflow scheduler
Date Thu, 06 Dec 2018 14:25:33 GMT
HI,

I have a query. It will be great if somebody in the community can clarify it.

Basically, for each dag_processing process, I can see the orm being reconfigured and disposed
towards the end. This is creating and closing a lot of connections on MySQL server, which
crashes in scenarios where there are a lot of new dags to process.

I want to know if this is the intended behavior and why? Also what kind of improvement can
be planned in this area so that we can reuse the connection pool amongst these processes.

Thanks,
Anand

Mime
View raw message