hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Update of "Hbase/HBaseTokenAuthentication" by GaryHelmling
Date Fri, 11 Mar 2011 20:14:55 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hbase/HBaseTokenAuthentication" page has been changed by GaryHelmling.
The comment on this change is: Add some details on ZK usage for key distribution.


   1. implement {{{org.apache.hadoop.security.token.SecretManager}}}
   1. master will generate a secret key for signing and authenticating tokens
     a. will need to persist somewhere (zookeeper?) to allow for master restarts and failover
+    a. the generated secret key will be distributed across all cluster nodes via ZooKeeper
+      i. ZooKeeper access to keys will be secure by Kerberos authentication (ZOOKEEPER-938)
and use of ACLs limiting access to HBase principals
-    a. will need to distribute generated secret key to RS
-      i. could be on region checkin/heartbeats, though stack is removing those
-      i. could be distributed through zookeeper as well
   1. add a helper like {{{TableMapReduceUtil.initJob()}}} to use when submitting a new job
     a. will obtain a new token from master
     a. add token to Credentials instance
@@ -53, +52 @@

  ==== Master Secret Key ====
  Authentication relies on a secret key generated at runtime on the master and used to generate
Authentication Tokens for clients.  Tokens will be generated on the master for Kerberos authenticated
clients, but token based authentication will need to be allowed on all masters and region
servers in a cluster.  So the master will need a means to distribute the secret key to other
cluster nodes.
- The master will also need to write the secret key to persistent storage in order for authentication
tokens to survive a cluster restart.
+ The master will also need to write the secret key to semi-persistent storage in order for
authentication tokens to survive a cluster restart.  The keys themselves are by nature transient,
as the current master key will be periodically rolled to limit exposure to reverse engineering
from token secrets.  The last N keys will be maintained in order to validate existing tokens,
using a fixed size queue, with the oldest key dropped on insertion when full.
+ ZooKeeper will be used to broadcast master key changes throughout the cluster and to provide
key persistence between master restarts or failover.  Note that this depends on securing access
to the key znodes via Kerberos authentication and ZooKeeper ACLs.  Keys will be stored in
ZooKeeper, with one znode per key, using the structure:
+ {{{
+       secretkey/
+           znode(keyID1, serialized DelegationKey1)
+           znode(keyID2, serialized DelegationKey2)
+           ...
+ }}}
  ==== Implementation ====
   1. Extend {{{org.apache.hadoop.security.token.TokenIdentifier}}} with new HBase type

View raw message