accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ke...@deenlo.com
Subject Re: Review Request 30817: ACCUMULO-3513 Delegation token support
Date Thu, 12 Feb 2015 14:58:02 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/30817/#review72159
-----------------------------------------------------------



docs/src/main/asciidoc/chapters/kerberos.txt
<https://reviews.apache.org/r/30817/#comment118163>

    This approach requires a user to modify the code for their map reduce job.  It would be
nice to have a way of doing this that did not require this modification.   This would probably
be a seperate issue.  Seems like it would require a standard way of serializing delegation
tokens and some command line tools to help produce those serialized tokens.


- kturner


On Feb. 11, 2015, 11:15 p.m., Josh Elser wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/30817/
> -----------------------------------------------------------
> 
> (Updated Feb. 11, 2015, 11:15 p.m.)
> 
> 
> Review request for accumulo and Christopher Tubbs.
> 
> 
> Bugs: ACCUMULO-3513
>     https://issues.apache.org/jira/browse/ACCUMULO-3513
> 
> 
> Repository: accumulo
> 
> 
> Description
> -------
> 
> DelegationTokens are a way to enable MapReduce jobs (or any distributed task which does
not have direct access to the user's kerberos credentials) to identify themself and communicate
with Accumulo. This is implemented in the same style of Hadoop which does this to provide
HDFS and YARN access in YARN jobs, as well as HBase (HBase also served as the base when I
started coding).
> 
> A DelegationToken (nothing more than a short-lived, on-demand password generated by Accumulo)
is provided to users who request one through the SecurityOperations API. Secret keys are created,
distributed and expired within Accumulo itself (Master creates and expires keys, distributes
via ZK to tservers), which allow the master to create delegation tokens upon request and tabletservers
to validate delegation tokens provided as the authentication mechanism by users. Tokens are
valid for 7 days by default and new secret keys are rolled every day (configurable).
> 
> RPC is implemented SASL's DIGEST-MD5 mechanism (in addition to the GSSAPI support we
already have for "normal" kerberos -- thrift makes this really easy). YARN provides a secure
way to pass delegation tokens from the client JVM to each YARN task (mapper/reducer) to prevent
unwanted parties from stealing the token (password) and using it for their own (nefarious)
purposes. The feature is transparent for users as long as they as using our MapReduce JCommander
options classes.
> 
> User documentation has been added to the kerberos chapter.
> 
> 
> Diffs
> -----
> 
>   core/src/main/java/org/apache/accumulo/core/Constants.java 0229d4e 
>   core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java 216f32d 
>   core/src/main/java/org/apache/accumulo/core/cli/MapReduceClientOnDefaultTable.java
0cf081f 
>   core/src/main/java/org/apache/accumulo/core/cli/MapReduceClientOnRequiredTable.java
7719e92 
>   core/src/main/java/org/apache/accumulo/core/cli/MapReduceClientOpts.java 4b3b7ed 
>   core/src/main/java/org/apache/accumulo/core/client/admin/SecurityOperations.java efeafc0

>   core/src/main/java/org/apache/accumulo/core/client/impl/ClientContext.java 8470da4

>   core/src/main/java/org/apache/accumulo/core/client/impl/SecurityOperationsImpl.java
feb1ee7 
>   core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportKey.java 072724b

>   core/src/main/java/org/apache/accumulo/core/client/mapred/AbstractInputFormat.java
b83a024 
>   core/src/main/java/org/apache/accumulo/core/client/mapred/AccumuloOutputFormat.java
f877ec6 
>   core/src/main/java/org/apache/accumulo/core/client/mapreduce/AbstractInputFormat.java
5c7b780 
>   core/src/main/java/org/apache/accumulo/core/client/mapreduce/AccumuloOutputFormat.java
5e0aa73 
>   core/src/main/java/org/apache/accumulo/core/client/mapreduce/impl/DelegationTokenStub.java
PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/impl/ConfiguratorBase.java
b2b5150 
>   core/src/main/java/org/apache/accumulo/core/client/mapreduce/lib/impl/InputConfigurator.java
5405ac0 
>   core/src/main/java/org/apache/accumulo/core/client/mock/MockSecurityOperations.java
db88cfb 
>   core/src/main/java/org/apache/accumulo/core/client/security/tokens/DelegationToken.java
PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/conf/Property.java 68fac73 
>   core/src/main/java/org/apache/accumulo/core/master/thrift/MasterClientService.java
4b90a34 
>   core/src/main/java/org/apache/accumulo/core/rpc/SaslClientDigestCallbackHandler.java
PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/rpc/SaslConnectionParams.java e067e23 
>   core/src/main/java/org/apache/accumulo/core/rpc/SaslDigestCallbackHandler.java PRE-CREATION

>   core/src/main/java/org/apache/accumulo/core/rpc/ThriftUtil.java d880fb3 
>   core/src/main/java/org/apache/accumulo/core/security/AuthenticationTokenIdentifier.java
PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/security/SystemPermission.java b998179

>   core/src/main/java/org/apache/accumulo/core/security/thrift/TAuthenticationKey.java
PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/security/thrift/TAuthenticationTokenIdentifier.java
PRE-CREATION 
>   core/src/main/java/org/apache/accumulo/core/security/thrift/TDelegationToken.java PRE-CREATION

>   core/src/main/java/org/apache/accumulo/core/util/ThriftMessageUtil.java PRE-CREATION

>   core/src/main/thrift/master.thrift d89e381 
>   core/src/main/thrift/security.thrift 66235a8 
>   core/src/test/java/org/apache/accumulo/core/client/impl/ThriftTransportKeyTest.java
2723273 
>   core/src/test/java/org/apache/accumulo/core/client/security/tokens/DelegationTokenTest.java
PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/rpc/SaslClientDigestCallbackHandlerTest.java
PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/rpc/SaslConnectionParamsTest.java 3910f34

>   core/src/test/java/org/apache/accumulo/core/security/AuthenticationTokenIdentifierTest.java
PRE-CREATION 
>   core/src/test/java/org/apache/accumulo/core/util/ThriftMessageUtilTest.java PRE-CREATION

>   docs/src/main/asciidoc/chapters/kerberos.txt 05d7384 
>   fate/src/main/java/org/apache/accumulo/fate/zookeeper/IZooReader.java 610b1bd 
>   fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooReader.java 5706cf3 
>   fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooUtil.java 805bfff 
>   proxy/src/main/java/org/apache/accumulo/proxy/Proxy.java f9039be 
>   server/base/src/main/java/org/apache/accumulo/server/AccumuloServerContext.java 84c3853

>   server/base/src/main/java/org/apache/accumulo/server/master/state/MetaDataStateStore.java
bf56a7a 
>   server/base/src/main/java/org/apache/accumulo/server/rpc/SaslServerConnectionParams.java
PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/rpc/SaslServerDigestCallbackHandler.java
PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingInvocationHandler.java
f85505d 
>   server/base/src/main/java/org/apache/accumulo/server/rpc/TServerUtils.java f1f8963

>   server/base/src/main/java/org/apache/accumulo/server/rpc/UGIAssumingProcessor.java
ab106a6 
>   server/base/src/main/java/org/apache/accumulo/server/security/AuditedSecurityOperation.java
cc7a7cd 
>   server/base/src/main/java/org/apache/accumulo/server/security/SecurityOperation.java
7adb46e 
>   server/base/src/main/java/org/apache/accumulo/server/security/SystemCredentials.java
51d50a1 
>   server/base/src/main/java/org/apache/accumulo/server/security/delegation/AuthenticationKey.java
PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenKeyManager.java
PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenSecretManager.java
PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyDistributor.java
PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyWatcher.java
PRE-CREATION 
>   server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosAuthenticator.java
08fa55b 
>   server/base/src/test/java/org/apache/accumulo/server/AccumuloServerContextTest.java
49a60a6 
>   server/base/src/test/java/org/apache/accumulo/server/rpc/SaslDigestCallbackHandlerTest.java
PRE-CREATION 
>   server/base/src/test/java/org/apache/accumulo/server/rpc/SaslServerConnectionParamsTest.java
PRE-CREATION 
>   server/base/src/test/java/org/apache/accumulo/server/security/delegation/AuthenticationKeyTest.java
PRE-CREATION 
>   server/base/src/test/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenKeyManagerTest.java
PRE-CREATION 
>   server/base/src/test/java/org/apache/accumulo/server/security/delegation/AuthenticationTokenSecretManagerTest.java
PRE-CREATION 
>   server/base/src/test/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyDistributorTest.java
PRE-CREATION 
>   server/base/src/test/java/org/apache/accumulo/server/security/delegation/ZooAuthenticationKeyWatcherTest.java
PRE-CREATION 
>   server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java da0b07c

>   server/gc/src/test/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogsTest.java
1d7f90f 
>   server/gc/src/test/java/org/apache/accumulo/gc/SimpleGarbageCollectorTest.java 6fcdd37

>   server/gc/src/test/java/org/apache/accumulo/gc/replication/CloseWriteAheadLogReferencesTest.java
120692a 
>   server/master/src/main/java/org/apache/accumulo/master/Master.java be476de 
>   server/master/src/main/java/org/apache/accumulo/master/MasterClientServiceHandler.java
72cba26 
>   server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java a5675dc

>   test/src/main/java/org/apache/accumulo/test/continuous/ContinuousBatchWalker.java a2687bb

>   test/src/main/java/org/apache/accumulo/test/continuous/ContinuousIngest.java dba6ac9

>   test/src/main/java/org/apache/accumulo/test/continuous/ContinuousMoru.java 4b5c3e7

>   test/src/main/java/org/apache/accumulo/test/continuous/ContinuousOpts.java PRE-CREATION

>   test/src/main/java/org/apache/accumulo/test/continuous/ContinuousQuery.java 73048f6

>   test/src/main/java/org/apache/accumulo/test/continuous/ContinuousScanner.java f68377a

>   test/src/main/java/org/apache/accumulo/test/continuous/ContinuousWalk.java 60f8ec2

>   test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java 06b4303 
>   test/src/test/java/org/apache/accumulo/test/ShellServerIT.java 9296548 
>   test/src/test/java/org/apache/accumulo/test/functional/KerberosIT.java 3d48657 
> 
> Diff: https://reviews.apache.org/r/30817/diff/
> 
> 
> Testing
> -------
> 
> Many new unit tests. Additional integration tests. Standalone mapreduce jobs (continuous
verify and terasort ingest) on a fully secure cluster.
> 
> 
> Thanks,
> 
> Josh Elser
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message