airflow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bolke de Bruin <bdbr...@gmail.com>
Subject Re: possible bug: scheduler dont schedule manual triggered dag on 1.8.b2
Date Wed, 25 Jan 2017 08:00:20 GMT
Thanks for tracking this down Jayesh. I suspect not wanting ‘backfills’ is the reason for
it (ie. backfills are not scheduled yet), but I need to look through the code. Maybe Dan or
Paul can chime in?

Bolke

> On 25 Jan 2017, at 02:46, Jayesh Senjaliya <jhsonline@gmail.com> wrote:
> 
> I am running airflow 1.8.b2
> 
> a week before I had perfectly running dag which i put it on 1.8.b2 and I
> find that scheduler is not scheduling the task even though the dag run was
> created.
> 
> I debugged the issue and came down to this piece of code where scheduler
> finds task to schedule
> 
> TI = models.TaskInstance
> task_instances_to_examine = (
>    session
>    .query(TI)
>    .filter(TI.dag_id.in_(simple_dag_bag.dag_ids))
>    .filter(TI.state.in_(states))
>    .join(DagRun, and_(TI.dag_id == DagRun.dag_id,
>                       TI.execution_date == DagRun.execution_date,
>                       DagRun.state == State.RUNNING,
>                       DagRun.run_id.like(DagRun.ID_PREFIX + '%')))
>    .all()
> )
> 
> 
> the root cause i think is join in the above query was added recently with
> this PR  "[AIRFLOW-219][AIRFLOW-398] Cgroups + impersonation"
> which has condition that run_id should be like DagRun.ID_PREFIX which is
> "scheduled__" which wont be true for manually triggered dag ( which has
> run_id like "manual__2017-01-24T17:08:35.251019" )
> 
> so I am not sure why we need that condition there at all, does anyone know?
> 
> Please comment.
> 
> Thanks
> Jayesh


Mime
View raw message