hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shane Kumpf (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6726) Fix issues with docker commands executed by container-executor
Date Tue, 11 Jul 2017 13:20:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16082179#comment-16082179
] 

Shane Kumpf commented on YARN-6726:
-----------------------------------

I've uploaded a patch to address the items above and a few more. The summary of changes and
why they are needed is as follows.

# {{docker stop, rm, kill}} - The final argument is the container name. Wrapping the container
name in single quotes results in Docker being unable to find the image. i.e. {{container_xyz}}
becomes {{'container_xyz'}} at invocation time.
# {{docker inspect}} - The --format option is used to obtain the container PID, hostname,
and IP address. Attempting to quote the inspect option does not work. The quotes then become
part of the output. i.e. {{'"'"'127.0.0.1,localhost.localdomain'"'"'}} would be returned if
the --format option is quoted. I've skipped quoting for this option. No user input is used,
so this should be safe.
# LOGFILE flush - The LOGFILE flush containing the command to be run is a problem. When PrivilegedOperationExecutor
is asked to return the operations output, this log entry becomes part of the output, breaking
commands such as {{docker inspect}}. Removed the flush.
# {{docker pull}} - Requires special handling. The only argument here is the image name, which
is user supplied. Quoting the image name results in the same issues seen for container names
in the {{docker stop, rm, kill}} case. To address this, if the subcommand is {{pull}} it is
expected that the next element in the array is the image name. The image name is then validated
against the same regex used in the runtime within Java. This does add a dependency on the
regex library, which has portability issues. I'm curious how we can address the portability
issue? Skipping validation on specific platforms seems inappropriate. I'm not sure only doing
the validation on the java side is enough, so skipping validation entirely also seems inappropriate.
Open to suggestions here.
# {{docker -- config=/path/to/conf/dir}} - Using a Docker client config requires the {{--config}}
option to come before any sub commands. If the first element in the array is this option,
pop it off the front of the array, add it to the output, and shift the array down. The current
approach being considered is that the Docker client config will be localized with the application,
and thus will already be checked to ensure it is a file. No user input will be allowed, only
shipping the config.json to be localized.

> Fix issues with docker commands executed by container-executor
> --------------------------------------------------------------
>
>                 Key: YARN-6726
>                 URL: https://issues.apache.org/jira/browse/YARN-6726
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Shane Kumpf
>            Assignee: Shane Kumpf
>         Attachments: YARN-6726.001.patch
>
>
> docker inspect, rm, stop, etc are issued through container-executor. Commands other than
docker run are not functioning properly.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message