Mailing-List: contact commits-help@airflow.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@airflow.incubator.apache.org
Date: Fri, 22 Sep 2017 19:55:00 +0000 (UTC)
From: "Gregory Benison (JIRA)" <jira@apache.org>
To: commits@airflow.incubator.apache.org
Message-ID: <JIRA.13104382.1506110089000.179424.1506110100377@Atlassian.JIRA>
In-Reply-To: <JIRA.13104382.1506110089000@Atlassian.JIRA>
References: <JIRA.13104382.1506110089000@Atlassian.JIRA> <JIRA.13104382.1506110089517@jira-lw-us.apache.org>
Subject: [jira] [Created] (AIRFLOW-1633) docker_operator needs a way to set
 shm_size
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 22 Sep 2017 19:55:05 -0000

Gregory Benison created AIRFLOW-1633:
----------------------------------------

             Summary: docker_operator needs a way to set shm_size
                 Key: AIRFLOW-1633
                 URL: https://issues.apache.org/jira/browse/AIRFLOW-1633
             Project: Apache Airflow
          Issue Type: Improvement
            Reporter: Gregory Benison


The "docker run" command line client has an option "--shm-size" that allows setting the size of /dev/shm in the container.  The "docker" package - on which docker_operator is based - does provide an interface for setting this property; docker.utils.create_host_config() has a "shm_size" argument that defaults to None.  However docker_operator calls that method in a way that does not provide a mechanism for accessing that argument:
{noformat}
            self.container = self.cli.create_container(
                    command=self.get_command(),
                    cpu_shares=cpu_shares,
                    environment=self.environment,
                    host_config=self.cli.create_host_config(binds=self.volumes,
                                                            network_mode=self.network_mode),
                    image=image,
                    mem_limit=self.mem_limit,
                    user=self.user
            )
{noformat}

It would be useful to provide a way to pass the "shm_size" option to the call to create_host_config().  Actually create_host_config() has a large number of options that can't be controlled through docker_operator - perhaps it would make sense to find a way to expose all or most of them.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)