airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kaxil Naik (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AIRFLOW-839) docker_operator.py attempts to log status key without first checking existence
Date Fri, 26 Oct 2018 09:26:01 GMT

     [ https://issues.apache.org/jira/browse/AIRFLOW-839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kaxil Naik updated AIRFLOW-839:
-------------------------------
    Fix Version/s:     (was: 1.10.1)
                   2.0.0

> docker_operator.py attempts to log status key without first checking existence
> ------------------------------------------------------------------------------
>
>                 Key: AIRFLOW-839
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-839
>             Project: Apache Airflow
>          Issue Type: Bug
>          Components: docker
>    Affects Versions: 1.7.1
>         Environment: arch linux
> python 2.7 and python 3
>            Reporter: Mike Perry
>            Assignee: Mike Perry
>            Priority: Minor
>             Fix For: 2.0.0
>
>
> When pulling a docker image, docker_operator.py attempts to log the `status` key each
time it gets output. This is usually fine, but occasionaly no `status` key exists. We've seen
this happen when we run out of inode space in our cluster and the docker cli is unable to
extract the image. This is consistent with the docker HTTP api docs (https://docs.docker.com/engine/api/v1.24/#create-an-image).
If an error occurs, there won't be a `status` key. There will be an `error` key. 
> This is a relatively minor bug, but it obscures the real issue and can sometimes make
it difficult to figure out what went wrong. 
> If you agree this should be fixed, I can submit a pr that first checks for status before
logging. 
> here's the code that would need changed: https://github.com/apache/incubator-airflow/blob/master/airflow/operators/docker_operator.py#L156
> Stack Trace
> [2017-01-10 22:47:21,980] {models.py:1286} ERROR - 'status'
> Traceback (most recent call last):
>   File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 1245, in run
>     result = task_copy.execute(context=context)
>   File "/usr/local/lib/python2.7/dist-packages/airflow/operators/docker_operator.py",
line 150, in execute
>     logging.info("{}".format(output['status']))
> KeyError: 'status'
> [2017-01-10 22:47:21,982] {models.py:1298} INFO - Marking task as UP_FOR_RETRY
> [2017-01-10 22:47:22,006] {models.py:1327} ERROR - 'status'
> [2017-01-10 22:47:22,704] {jobs.py:159} DEBUG - [heart] Boom.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message