airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ludovic Claude (JIRA)" <j...@apache.org>
Subject [jira] [Created] (AIRFLOW-944) Docker operator does not work with Docker >= 1.19
Date Mon, 06 Mar 2017 14:06:32 GMT
Ludovic Claude created AIRFLOW-944:
--------------------------------------

             Summary: Docker operator does not work with Docker >= 1.19
                 Key: AIRFLOW-944
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-944
             Project: Apache Airflow
          Issue Type: Bug
          Components: docker
    Affects Versions: Airflow 1.7.1.3, 1.8.0rc4
         Environment: Ubuntu 16.04
            Reporter: Ludovic Claude


Docker operator does not work when mem_limit is set and Docker version 1.19 or more recent
is used.

Here are the logs, I have seen this issue with Airflow 1.7.1.3 and Airflow 1.8.0 rc4.

[2017-03-06 11:37:54,895] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,895]
{docker_operator.py:132} INFO - Starting docker container from image hbpmip/mipmap
[2017-03-06 11:37:54,903] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,902]
{models.py:1417} ERROR - mem_limit has been moved to
 host_config in API version 1.19
[2017-03-06 11:37:54,903] {base_task_runner.py:95} INFO - Subtask: Traceback (most recent
call last):
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.5/dist-packages/airflow/models.py",
line 1369, in run
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:     result = task_copy.execute(context=context)
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:   File "/tmp/src/airflow-imaging-plugins/airflow_pipeline/operators/docker_pipeline_operator.py",
line 191, in execute
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:     logs = super(DockerPipelineOperator,
self).execute(context)
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.5/dist-packages/airflow/operators/docker_operator.py",
line 172, in execute
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:     user=self.user
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.5/dist-packages/docker/api/container.py",
line 133, in create_container
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:     volume_driver, stop_signal,
networking_config,
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.5/dist-packages/docker/api/container.py",
line 138, in create_container_config
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:     return utils.create_container_config(self._version,
*args, **kwargs)
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:   File "/usr/local/lib/python3.5/dist-packages/docker/utils/utils.py",
line 1041, in create_container_config
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask:     'mem_limit has been
moved to host_config in API version 1.19'
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: docker.errors.InvalidVersion:
mem_limit has been moved to host_config in API version 1.19
[2017-03-06 11:37:54,904] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,903]
{models.py:1433} INFO - Marking task as UP_FOR_RETRY
[2017-03-06 11:37:54,912] {base_task_runner.py:95} INFO - Subtask: [2017-03-06 11:37:54,912]
{models.py:1462} ERROR - mem_limit has been moved to host_config in API version 1.19




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message