hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3943) RM-NM secret-keys should be randomly generated and rolled every so often
Date Thu, 10 May 2012 20:56:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13272751#comment-13272751

Siddharth Seth commented on MAPREDUCE-3943:

Thanks for taking a look Daryn.

The patch requires MAPREDUCE-3940 to be applied first (if that still applies).

bq. It feels a bit contorted for the RM to have a pb message with the current and prior key
(ie. it's limited), which is the root of the 2X key roll problem. With the patch the way it
is, having the RM transmit a single key and the NM remembering N-many keys is probably "less
Current and prior keys are only sent during nodeRegistration - in case a node registers in
the window where the RM has generated a new key but hasn't started using it yet. nodeHeartbeat
calls only send the latest key.
The NM remembers 2 keys - which is what causes the 2x key roll problem. Remembering additional
keys on the NM needs to be controlled - older keys can't be used to startContainers. The half
patch I mentioned earlier - would get the NM to remember additional keys for containers which
have already been started. Will incorporate that into this patch.

bq. Passing the shared secret keys in "plaintext" in heartbeats is a bit troubling in general.
More concerning is the direction of the data flow: RM generates secret and gives it to the
NMs. A rogue or compromised NM can intercept a key which I believe can be used to generate
tokens for other NMs. If true, doesn't that put the entire cluster at risk?

bq. Conceptually, the RM should simply request a token from the NM and pass the token along
to the AM so it can contact the NM. It that's too expensive, it seems like the key exchange
should be inverted: NMs generate their own secret, and provide that secret to the RM. A compromised
node cannot damage the entire cluster.

Having NMs generate keys can get expensive.NMs generate keys and provide them to the RM can
be achieved by having the RM generate individual keys per NM. I'd prefer dealing with that
in a separate jira - since it's a reasonably independent change.

> RM-NM secret-keys should be randomly generated and rolled every so often
> ------------------------------------------------------------------------
>                 Key: MAPREDUCE-3943
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3943
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2, security
>    Affects Versions: 0.23.0
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>         Attachments: MAPREDUCE-3943-20120416.txt, MR3943.txt
>  - RM should generate the master-key randomly
>  - The master-key should roll every so often
>  - NM should remember old expired keys so that already doled out container-requests can
be satisfied.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message