hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7654) Support ENTRY_POINT for docker container
Date Thu, 10 May 2018 20:41:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16471073#comment-16471073
] 

Eric Yang commented on YARN-7654:
---------------------------------

[~jlowe] I am struggling withe the following problems:
{quote}AbstractProviderService#buildContainerLaunchContext so the pieces needed by DockerProviderService
can be reused without requiring the launcher command to be clobbered afterwards?{quote}

Launch command is override to bash -c 'launch-command' in DockerLinuxContainerRuntime, and
subsequently appended the log redirection. '2> <LOG_DIR>/stderr.txt 1> <LOG_DIR>/stdout.txt',
then replaced <LOG_DIR> with actual container logging directory.  The number of steps
to go through the preprocessing before writing to .cmd file complicates how to refactor the
code base without breaking things.  This is the reason that setCommand was created to flush
out the override commands to ensure the command is not tempered incorrectly during the hand
off from DockerLinuxContainerRuntime to DockerClient to container-executor.  For safety reason,
I keep setCommand to ensure the command is not tempered by string substitutions, and not break
YARN v2 API. 

{quote}The instance checking and downcasting in writeCommandToTempFile looks pretty ugly.
It would be cleaner to encapsulate this in the DockerCommand abstraction. One example way
to do this is to move the logic of writing a docker command file into the DockerCommand abstract
class. DockerRunCommand can then override that method to call the parent method and then separately
write the env file. Worst case we can add a getEnv method to DockerCommand that returns the
collection of environment variables to write out for a command. DockerCommand would return
null or an empty collection while DockerRunCommand can return its environment.{quote}

DockerCommand is a data structure class.  It does not handle IO operation.  If we move IO
operation to this class, it would not be clean data structure to represent the docker command.
 I think it is more self explanatory that for DockerRunCommand, we also write out the environment
file.  With changes in YARN-8261, we are interested to ensure that directory is created, create
the cmd file, create the env file.  For safety reason, I think we should not make the styling
changes for this area at this time because we are out of time to throughly retest what have
been tested in the previous patch set.



> Support ENTRY_POINT for docker container
> ----------------------------------------
>
>                 Key: YARN-7654
>                 URL: https://issues.apache.org/jira/browse/YARN-7654
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>    Affects Versions: 3.1.0
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Blocker
>              Labels: Docker
>         Attachments: YARN-7654.001.patch, YARN-7654.002.patch, YARN-7654.003.patch, YARN-7654.004.patch,
YARN-7654.005.patch, YARN-7654.006.patch, YARN-7654.007.patch, YARN-7654.008.patch, YARN-7654.009.patch,
YARN-7654.010.patch, YARN-7654.011.patch, YARN-7654.012.patch, YARN-7654.013.patch, YARN-7654.014.patch,
YARN-7654.015.patch, YARN-7654.016.patch, YARN-7654.017.patch, YARN-7654.018.patch, YARN-7654.019.patch,
YARN-7654.020.patch, YARN-7654.021.patch
>
>
> Docker image may have ENTRY_POINT predefined, but this is not supported in the current
implementation.  It would be nice if we can detect existence of {{launch_command}} and base
on this variable launch docker container in different ways:
> h3. Launch command exists
> {code}
> docker run [image]:[version]
> docker exec [container_id] [launch_command]
> {code}
> h3. Use ENTRY_POINT
> {code}
> docker run [image]:[version]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message