airflow-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF subversion and git services (JIRA)" <>
Subject [jira] [Commented] (AIRFLOW-1633) docker_operator needs a way to set shm_size
Date Tue, 10 Apr 2018 07:18:00 GMT


ASF subversion and git services commented on AIRFLOW-1633:

Commit e50eea629506bcafc5f2778eb3d0dfe0faf2670c in incubator-airflow's branch refs/heads/master
from Tao feng
[;h=e50eea6 ]

[AIRFLOW-1633] docker_operator needs a way to set shm_size

Closes #3199 from feng-tao/airflow_1633

> docker_operator needs a way to set shm_size
> -------------------------------------------
>                 Key: AIRFLOW-1633
>                 URL:
>             Project: Apache Airflow
>          Issue Type: Improvement
>            Reporter: Gregory Benison
>            Assignee: Tao Feng
>            Priority: Major
>             Fix For: 2.0.0
> The "docker run" command line client has an option "--shm-size" that allows setting the
size of /dev/shm in the container.  The "docker" package - on which docker_operator is based
- does provide an interface for setting this property; docker.utils.create_host_config() has
a "shm_size" argument that defaults to None.  However docker_operator calls that method in
a way that does not provide a mechanism for accessing that argument:
> {noformat}
>             self.container = self.cli.create_container(
>                     command=self.get_command(),
>                     cpu_shares=cpu_shares,
>                     environment=self.environment,
>                     host_config=self.cli.create_host_config(binds=self.volumes,
>                                                             network_mode=self.network_mode),
>                     image=image,
>                     mem_limit=self.mem_limit,
>                     user=self.user
>             )
> {noformat}
> It would be useful to provide a way to pass the "shm_size" option to the call to create_host_config().
 Actually create_host_config() has a large number of options that can't be controlled through
docker_operator - perhaps it would make sense to find a way to expose all or most of them.

This message was sent by Atlassian JIRA

View raw message