airflow-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandeep S <3005sand...@gmail.com>
Subject Re: Issues running only one active instance in a DAG
Date Fri, 02 Oct 2020 12:02:04 GMT
Hi All,

I am having a production issue running only one instance of DAG at a time.
If the DAG is running one instance, a 2nd instance does not kick off. But
if any of the task fails in the active DAG instance, the DAG gets marked
failed but a 2nd instance kicks off after 5 mins(5 mins scheduled time for
DAG.

Please help.

Regards
Sandeep

On Mon, Sep 28, 2020 at 1:18 PM Tavares Forby <tforby@qti.qualcomm.com>
wrote:

>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Hi All,
>
>
>
>
>
>
>
>
>
>
>
> I am having a few issues with Airflow and task instances greater than
> 750.  I am getting one consistent error and one error that happens random
> (understand, it's technically not random).
>
>
>
>
>
>
>
>
>
>
>
> Consistent error:
>
>
>
>
> [2020-09-25 12:28:01,703] {scheduler_job.py:237}
>
> WARNING - Killing PID 119970 [2020-09-25 12:29:17,110]
> {scheduler_job.py:237} WARNING - Killing PID 121013 [2020-09-25
> 12:29:17,110] {scheduler_job.py:237} WARNING - Killing PID 121013
> [2020-09-25 12:30:12,171] {scheduler_job.py:237} WARNING - Killing PID
>
> 123243
>
>
>
>
>
>
>
>
>
>
>
>
>
> Random error:
>
>
>
>
>  [2020-09-27
>
> 19:37:25,127] {scheduler_job.py:771} INFO - Examining DAG run <DagRun
> tutorial_large_design_debug7 @ 2020-09-28 02:37:24+00:00:
> manual__2020-09-28T02:37:24+00:00, externally triggered: True> [2020-09-27
> 19:37:26,749] {logging_mixin.py:112} INFO - [2020-09-27
>
> 19:37:26,749] {dagrun.py:408} INFO - (MySQLdb.
>
> *exceptions.IntegrityError) (1062, "Duplicate entry 'echo__a*-tutorial_large_design_debug7-2020-
> 09-28 02:37:24.000000' for key 'PRIMARY'")
>
> [SQL: INSERT INTO task_instance (task_id, dag_id, execution_date,
> start_date, end_date, duration, state, try_number, max_tries, hostname,
> unixname, job_id, pool, pool_slots, queue, priority_weight, operator, qu
> eued_dttm, pid, executor_config) VALUES (%s,
>
> %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s,
> %s)]
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Please
>
> help! thanks!
>
>
>
>
>
>
>

Mime
View raw message