accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser" <josh.el...@gmail.com>
Subject Re: Review Request 29386: ACCUMULO-2815 Client authentication via Kerberos
Date Thu, 08 Jan 2015 23:52:21 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/29386/
-----------------------------------------------------------

(Updated Jan. 8, 2015, 11:52 p.m.)


Review request for accumulo.


Changes
-------

Addresses the last major open issues:

* Removes use of shortUserName as the accumulo principal. The Accumulo principal is now the
same as the KRB principal -- we make no distinction.
* Adds user manual chapter: overview, client configuration/use and server configuration/use.
* Introduces KerberosAuthorizor and KerberosPermissionHandler. The use of KRB principal as
our principal means that principals can now contain "/" (e.g. `myserver/hostname@REALM`) which
doesn't function as expected with ZK. Principals (user parent ZNode) are base64 encoded to
work around this. The Authorizor and PermissionHandler are still essentially shims around
the ZK variants. This is done at the cost of human readable ZK nodes; however, this was already
a read-ACL'ed section of ZK, so people couldn't get in there on their own. I'm not sure if
there is anything else we could actually do to avoid this (it didn't seem like I could insert
escape chars).


I believe I am happy with the current state of this, so barring any negative feedback or requested
changes, I'll be applying this one soon. Please let me know (here or in private), if you'd
like cycles to review before I apply. As always, I'm happy to entertain continued review even
after this is applied.


Bugs: ACCUMULO-2815
    https://issues.apache.org/jira/browse/ACCUMULO-2815


Repository: accumulo


Description
-------

ACCUMULO-2815 Initial support for Kerberos client authentication.

Leverage SASL transport provided by Thrift which can speak GSSAPI, which Kerberos implements.
Introduced...

* An Accumulo KerberosToken which is an AuthenticationToken to validate users.
* Custom thrift processor and invocation handler to ensure server RPCs have a valid KRB identity
and Accumulo authentication.
* A KerberosAuthenticator which extends ZKAuthenticator to support Kerberos identities seamlessly.
* New ClientConf variables to use SASL transport and pass Kerberos server principal
* Updated ClientOpts and Shell opts to transparently use a KerberosToken when SASL is enabled
(no extra client work).

I believe this is the "bare minimum" for Kerberos support. They are also grossly lacking in
unit and integration tests. I believe that I might have somehow broken the client address
string in the server (I saw log messages with client: null, but I'm not sure if it's due to
these changes or not). A necessary limitation in the Thrift server used is that, like the
SSL transport, the SASL transport cannot presently be used with the TFramedTransport, which
means none of the [half]async thrift servers will function with this -- we're stuck with the
TThreadPoolServer.

Performed some contrived benchmarks on my laptop (while still using it myself) to get at big-picture
view of the performance impact against "normal" operation and Kerberos alone. Each "run" was
the duration to ingest 100M records using continuous-ingest, timed with `time`, using 'real'.

THsHaServer (our default), 6 runs:

Avg: 10m7.273s (607.273s)
Min: 9m43.395s
Max: 10m52.715s

TThreadPoolServer (no SASL), 5 runs:

Avg: 11m16.254s (676.254s)
Min: 10m30.987s
Max: 12m24.192s

TThreadPoolServer+SASL/GSSAPI (these changes), 6 runs:

Avg: 13m17.187s (797.187s)
Min: 10m52.997s
Max: 16m0.975s

The general takeway is that there's about 15% performance degredation in its initial state
which is in the realm of what I expected (~10%).


Diffs (updated)
-----

  README ad6f2bf 
  core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java f6ea934 
  core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java 6fe61a5 
  core/src/main/java/org/apache/accumulo/core/client/impl/ClientContext.java e75bec6 
  core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java f481cc3 
  core/src/main/java/org/apache/accumulo/core/client/impl/MasterClient.java a9ad8a1 
  core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportKey.java 6dc846f

  core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportPool.java 5da803b

  core/src/main/java/org/apache/accumulo/core/client/security/tokens/KerberosToken.java PRE-CREATION

  core/src/main/java/org/apache/accumulo/core/conf/Property.java e054a5f 
  core/src/main/java/org/apache/accumulo/core/rpc/FilterTransport.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/rpc/SaslConnectionParams.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/rpc/TTimeoutTransport.java 6eace77 
  core/src/main/java/org/apache/accumulo/core/rpc/ThriftUtil.java 09bd6c4 
  core/src/main/java/org/apache/accumulo/core/rpc/UGIAssumingTransport.java PRE-CREATION 
  core/src/main/java/org/apache/accumulo/core/rpc/UGIAssumingTransportFactory.java PRE-CREATION

  core/src/main/java/org/apache/accumulo/core/security/Credentials.java 525a958 
  core/src/test/java/org/apache/accumulo/core/cli/TestClientOpts.java ff49bc0 
  core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java PRE-CREATION

  core/src/test/java/org/apache/accumulo/core/client/impl/ThriftTransportKeyTest.java PRE-CREATION

  core/src/test/java/org/apache/accumulo/core/conf/ClientConfigurationTest.java 40be70f 
  core/src/test/java/org/apache/accumulo/core/rpc/SaslConnectionParamsTest.java PRE-CREATION

  docs/src/main/asciidoc/accumulo_user_manual.asciidoc ec8e538 
  docs/src/main/asciidoc/chapters/clients.txt 64f0e55 
  docs/src/main/asciidoc/chapters/kerberos.txt PRE-CREATION 
  minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloClusterImpl.java
27d6b19 
  minicluster/src/main/java/org/apache/accumulo/minicluster/impl/MiniAccumuloConfigImpl.java
26c23ed 
  pom.xml ae188a0 
  proxy/src/main/java/org/apache/accumulo/proxy/Proxy.java 4b048eb 
  server/base/src/main/java/org/apache/accumulo/server/AccumuloServerContext.java 09ae4f4

  server/base/src/main/java/org/apache/accumulo/server/init/Initialize.java 046cfb5 
  server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingInvocationHandler.java
PRE-CREATION 
  server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingWrapper.java
PRE-CREATION 
  server/base/src/main/java/org/apache/accumulo/server/rpc/TServerUtils.java 641c0bf 
  server/base/src/main/java/org/apache/accumulo/server/rpc/ThriftServerType.java PRE-CREATION

  server/base/src/main/java/org/apache/accumulo/server/security/SecurityOperation.java 5e81018

  server/base/src/main/java/org/apache/accumulo/server/security/SecurityUtil.java 29e4939

  server/base/src/main/java/org/apache/accumulo/server/security/SystemCredentials.java a59d57c

  server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosAuthenticator.java
PRE-CREATION 
  server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosAuthorizor.java
PRE-CREATION 
  server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosPermissionHandler.java
PRE-CREATION 
  server/base/src/main/java/org/apache/accumulo/server/thrift/UGIAssumingProcessor.java PRE-CREATION

  server/base/src/main/java/org/apache/accumulo/server/util/Admin.java ae36f1f 
  server/base/src/main/java/org/apache/accumulo/server/util/ZooZap.java 7fdbf13 
  server/base/src/test/java/org/apache/accumulo/server/AccumuloServerContextTest.java PRE-CREATION

  server/base/src/test/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingInvocationHandlerTest.java
PRE-CREATION 
  server/base/src/test/java/org/apache/accumulo/server/rpc/ThriftServerTypeTest.java PRE-CREATION

  server/base/src/test/java/org/apache/accumulo/server/security/SystemCredentialsTest.java
4202a7e 
  server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 93a9a49 
  server/gc/src/test/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogsTest.java f98721f

  server/gc/src/test/java/org/apache/accumulo/gc/SimpleGarbageCollectorTest.java 99558b8 
  server/gc/src/test/java/org/apache/accumulo/gc/replication/CloseWriteAheadLogReferencesTest.java
cad1e01 
  server/master/src/main/java/org/apache/accumulo/master/Master.java 12195fa 
  server/monitor/src/main/java/org/apache/accumulo/monitor/servlets/trace/Basic.java 2d98fed

  server/tracer/src/main/java/org/apache/accumulo/tracer/TraceServer.java 7e33300 
  server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java d5c1d2f 
  server/tserver/src/main/java/org/apache/accumulo/tserver/replication/ReplicationWorker.java
1d20e2b 
  shell/src/main/java/org/apache/accumulo/shell/Shell.java 58308ff 
  shell/src/main/java/org/apache/accumulo/shell/ShellOptionsJC.java 8167ef8 
  shell/src/test/java/org/apache/accumulo/shell/ShellConfigTest.java 0e72c8c 
  shell/src/test/java/org/apache/accumulo/shell/ShellOptionsJCTest.java PRE-CREATION 
  test/pom.xml b0a926f 
  test/src/main/java/org/apache/accumulo/test/functional/ZombieTServer.java eb84533 
  test/src/main/java/org/apache/accumulo/test/performance/thrift/NullTserver.java 2ebc2e3

  test/src/test/java/org/apache/accumulo/harness/AccumuloClusterIT.java 8f7e1b7 
  test/src/test/java/org/apache/accumulo/harness/MiniClusterHarness.java abdb627 
  test/src/test/java/org/apache/accumulo/harness/SharedMiniClusterIT.java 2380f66 
  test/src/test/java/org/apache/accumulo/harness/TestingKdc.java PRE-CREATION 
  test/src/test/java/org/apache/accumulo/harness/conf/AccumuloMiniClusterConfiguration.java
11b7530 
  test/src/test/java/org/apache/accumulo/server/security/SystemCredentialsIT.java fb71f5f

  test/src/test/java/org/apache/accumulo/test/ArbitraryTablePropertiesIT.java aa5c164 
  test/src/test/java/org/apache/accumulo/test/CleanWalIT.java 1fcd5a4 
  test/src/test/java/org/apache/accumulo/test/functional/BatchScanSplitIT.java 221889b 
  test/src/test/java/org/apache/accumulo/test/functional/KerberosIT.java PRE-CREATION 
  test/src/test/java/org/apache/accumulo/test/security/KerberosTokenTest.java PRE-CREATION

  test/src/test/resources/log4j.properties cb35840 

Diff: https://reviews.apache.org/r/29386/diff/


Testing
-------

Ensure existing unit tests still function. Accumulo is functional and ran continuous ingest
multiple times using a client with only a Kerberos identity (no user/password provided). Used
MIT Kerberos with Apache Hadoop 2.6.0 and Apache ZooKeeper 3.4.5.


Thanks,

Josh Elser


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message