airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "t oo (Jira)" <j...@apache.org>
Subject [jira] [Closed] (AIRFLOW-6437) sql filters - remove in (NULL)
Date Mon, 06 Jan 2020 21:00:00 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

t oo closed AIRFLOW-6437.
-------------------------
    Fix Version/s: master
                   2.0.0
       Resolution: Fixed

> sql filters - remove in (NULL)
> ------------------------------
>
>                 Key: AIRFLOW-6437
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-6437
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: core
>    Affects Versions: 1.10.7
>            Reporter: t oo
>            Assignee: t oo
>            Priority: Trivial
>             Fix For: 2.0.0, master
>
>
> looking at sql generated by sqlalchemy, example query:
> 2020-01-03 09:10:25,373 INFO sqlalchemy.engine.base.Engine SELECT task_instance.try_number
AS task_instance_try_number, task_instance.task_id AS task_instance_task_id, task_instance.dag_id
AS task_instance_dag_id, task_instance.execution_date AS task_instance_execution_date, task_instance.start_date
AS task_instance_start_date, task_instance.end_date AS task_instance_end_date, task_instance.duration
AS task_instance_duration, task_instance.state AS task_instance_state, task_instance.max_tries
AS task_instance_max_tries, task_instance.hostname AS task_instance_hostname, task_instance.unixname
AS task_instance_unixname, task_instance.job_id AS task_instance_job_id, task_instance.pool
AS task_instance_pool, task_instance.queue AS task_instance_queue, task_instance.priority_weight
AS task_instance_priority_weight, task_instance.operator AS task_instance_operator, task_instance.queued_dttm
AS task_instance_queued_dttm, task_instance.pid AS task_instance_pid, task_instance.executor_config
AS task_instance_executor_config 
> FROM task_instance 
> WHERE task_instance.dag_id = ? AND task_instance.execution_date = ? AND (task_instance.state
IN (?, *NULL*, ?) OR task_instance.state IS NULL)
> 2020-01-03 09:10:25,374 INFO sqlalchemy.engine.base.Engine ('example_bash_operator',
'2020-01-01 00:00:00.000000', 'up_for_reschedule', 'up_for_retry')
> the bolded part should not go into the query
> fix is to change below parts to rewrite the states list to not have null before passing
to the query
> grep 'if None in' -R *
> airflow/models/dagrun.py:                if None in state:
> airflow/models/dag.py:                if None in state:
> airflow/models/dag.py:            if None in states:
> airflow/jobs/scheduler_job.py:        if None in states:
> airflow/jobs/scheduler_job.py:        if None in acceptable_states:



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message