hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container
Date Thu, 02 May 2013 04:02:16 GMT

    [ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13647263#comment-13647263

Vinod Kumar Vavilapalli commented on YARN-613:

bq. I just have general concerns with assuming the entire hadoop environment is trusted and
thus introducing weaknesses at a global level . Ex. A weakness is introduced every time one
entity shares a secret to validate a token created by another entity. Compromising one of
hundreds or thousands of node shouldn't put the entire cluster at risk.
Agree with you in general. Read on.

bq. If I can gain access to one NM host and its keytab, I believe I can secretly launch a
malicious NM?
That is true in general.  And I am not sure how we can even contain such a break-in. I suppose
going the way of DataNode to start the server on privileged ports will contain it [1]. If
one can get hold of the keytab(owned by YARN user), I suppose at that point he can launch
the container-executor binary too, which will give him root access. So it's all predicated
on secure setup to not do stupid things.

bq. NMs currently share a global key container token secrets, but there is a jira to move
to per-NM secrets so sharing a global AM secret would be another step backwards.

bq. Exploring alternate avenues to avoid global trust, is passing the allowed am token allowed
to get status and stop the container with the launch request not feasible?
May be it isn't clear in my proposal, but let me state it again anyways, mostly repeating
what I just commented about on YARN-617.
 - Having the authentication via container-token is forcing us to create a connection per-container.
 - MR's ContainerLauncher for example resorts to tricks like creating lots of threads, opening
and closing connections immediately to avoid hitting ulimits etc.
 - Most of that ugliness will go away if we perform all authentication using AMTokens for
*all* AM-NM APIs and use ContainerTokens for authorization of startContainer() requests.

May be we should just do [1] above (previleged ports).

To sum it up, I am open to suggestions. My fundamental requirements are:
 - If possible, AMs should open only one connection - secure one - to each NM. Not one per
 - All connections (all APIs) between AM and NM should be authenticated - DIGEST based at
best here - and if possible without AMs having to latch on to things like ContainerTokens
for potentially long periods.
> Create NM proxy per NM instead of per container
> -----------------------------------------------
>                 Key: YARN-613
>                 URL: https://issues.apache.org/jira/browse/YARN-613
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Vinod Kumar Vavilapalli
> Currently a new NM proxy has to be created per container since the secure authentication
is using a containertoken from the container.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message