hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7654) Support ENTRY_POINT for docker container
Date Thu, 15 Mar 2018 16:33:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16400676#comment-16400676

Jason Lowe commented on YARN-7654:

Thanks for the patch!  Switching the code to execv docker instead of popen docker is going
to be a fairly extensive change that isn't directly tied to supporting an entry point.  Would
it make more sense to track the execv effort as a separate JIRA?

What base should be used for the patch?  I couldn't apply it to trunk, branch-3.1, or branch-3.0.

bq.  I am currently running into a problem that after docker is started, the existing logic
for detecting docker exit is not working.

I couldn't tell from the patch what the problem would be.  If I can get the patch applied
and some more details on how it behaves when it doesn't work I may be able to help debug.

bq.  I am concerned that we are using popen for docker launch for non-entry point version,
it actually spawn a shell underneath.

Yes, popen opens up the possibility of the shell doing us "favors" that we really do not want.
 It would be safer to avoid it.

bq. We might need to convert the output buffer from char array to char pointer array to avoid
chopping strings in the middle.

This is a necessity.  By serializing multiple values into a String and then needing to rediscover
the individual values later, we're opening up the possibility that the arguments will be mishandled
based on the contents of the arguments accidentally looking like separator characters.  That's
one of the many dangers of running the shell instead of execv directly which we're trying
to avoid.  If we're going to be serious about leveraging execv for security and correctness
in light of untrusted data in the arguments then building up the command line for execv shouldn't
be serialzing and deserializing through String.

As a bonus for going through execv directly, the whole quoting and appending and sanitizing
argument logic seems unnecessary at that point.  We can just pass these values straight through
to the docker command for interpretation by Docker rather than worrying about how to quote
them properly so the shell doesn't butcher them on the way through.

Why does the code not check for an execv error and instead exits with a success code?  It
should log the strerror of the execv failure and exit with a failure code instead.

> Support ENTRY_POINT for docker container
> ----------------------------------------
>                 Key: YARN-7654
>                 URL: https://issues.apache.org/jira/browse/YARN-7654
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>    Affects Versions: 3.1.0
>            Reporter: Eric Yang
>            Assignee: Eric Yang
>            Priority: Blocker
>         Attachments: YARN-7654.001.patch, YARN-7654.002.patch
> Docker image may have ENTRY_POINT predefined, but this is not supported in the current
implementation.  It would be nice if we can detect existence of {{launch_command}} and base
on this variable launch docker container in different ways:
> h3. Launch command exists
> {code}
> docker run [image]:[version]
> docker exec [container_id] [launch_command]
> {code}
> {code}
> docker run [image]:[version]
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message