hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2856) Fix block protocol so that Datanodes don't require root or jsvc
Date Wed, 30 Apr 2014 19:18:23 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth updated HDFS-2856:

    Attachment: HDFS-2856.prototype.patch

It's been a while since we've discussed this one, so here is a recap.  We (the names listed
in the design doc) proposed introducing challenge-response authentication on DataTransferProtocol
based on exchanging a digest calculated using the block access token as a shared secret. 
This would establish mutual authentication between client and DataNode before tokens were
exchanged, and thus it would remove the requirement to launch as root and bind to a privileged
port.  There were a few rounds of feedback discussing exactly which pieces of data to feed
into the digest calculation.  [~atm] also suggested folding this into the SASL handshake he
had implemented for DataTransferProtocol encryption in HDFS-3637.

I'm attaching a prototype patch.  This is not intended to be committed.  It's just a high-level
demonstration intended to revive discussion on this issue.

The suggestion to fold this into the SASL handshake makes sense, because we can rely on the
existing DIGEST-MD5 mechanism to handle verifying the digests.  This means the scope of this
issue is about adding support for the full range of SASL QOPs on DataTransferProtocol.  We
already support auth-conf, and now we need to add support for auth and auth-int.

The patch demonstrates this by hacking on the existing {{DataTransferEncryptor}} code.  I
changed the configured QOP to auth and changed the password calculation to use the block access
token's password + the target DataNode's UUID + a client-supplied request timestamp.  I tested
this manually end-to-end.  (I needed to set {{dfs.encrypt.data.transfer}} to {{true}} to trigger
the code, but it's not really encrypting.)  I ran tcpdump while reading a file, and I confirmed
that the SASL negotiation is using auth for the QOP, no cipher parameter (so no encryption),
and the block content is passed unencrypted on the wire.

Early feedback is welcome.  There is still a lot of work remaining: renegotiating SASL between
multiple block ops with different tokens, reconciling all of this code against the existing
HDFS-3637 code, actually removing the privileged port restriction, and automated tests.

> Fix block protocol so that Datanodes don't require root or jsvc
> ---------------------------------------------------------------
>                 Key: HDFS-2856
>                 URL: https://issues.apache.org/jira/browse/HDFS-2856
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: datanode, security
>            Reporter: Owen O'Malley
>            Assignee: Chris Nauroth
>         Attachments: Datanode-Security-Design.pdf, Datanode-Security-Design.pdf, Datanode-Security-Design.pdf,
> Since we send the block tokens unencrypted to the datanode, we currently start the datanode
as root using jsvc and get a secure (< 1024) port.
> If we have the datanode generate a nonce and send it on the connection and the sends
an hmac of the nonce back instead of the block token it won't reveal any secrets. Thus, we
wouldn't require a secure port and would not require root or jsvc.

This message was sent by Atlassian JIRA

View raw message