airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yongjun Park <thehp.key...@gmail.com>
Subject Too many task instances
Date Wed, 19 Apr 2017 12:41:15 GMT
Hi folks.

I have a question about task instances.

Is it possible to delete old task instances that have run successfully?
Isn't it trying to backfill missing tasks?

I have about 1,500 dags and am getting more dags. There're about 300
thousand of task instances currently. 10,000 tasks instances are made by
every day. It'll use 3.6 million rows of mysql table in an year.

I have concerns about a table which stores task instances that makes large
table which can cause performance degradation.

How can I keep the table which stores task instances not to be bloated?


Thanks,
Yongjun

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message