Return-Path: X-Original-To: apmail-accumulo-dev-archive@www.apache.org Delivered-To: apmail-accumulo-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5598A10C18 for ; Mon, 5 Jan 2015 20:46:31 +0000 (UTC) Received: (qmail 32657 invoked by uid 500); 5 Jan 2015 20:46:32 -0000 Delivered-To: apmail-accumulo-dev-archive@accumulo.apache.org Received: (qmail 32612 invoked by uid 500); 5 Jan 2015 20:46:32 -0000 Mailing-List: contact dev-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list dev@accumulo.apache.org Received: (qmail 32587 invoked by uid 99); 5 Jan 2015 20:46:30 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Jan 2015 20:46:30 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 320A31CABF7; Mon, 5 Jan 2015 20:46:28 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============0091702226384659264==" MIME-Version: 1.0 Subject: Re: Review Request 29386: ACCUMULO-2815 Client authentication via Kerberos From: "Christopher Tubbs" To: "Christopher Tubbs" , "Josh Elser" , "accumulo" Date: Mon, 05 Jan 2015 20:46:28 -0000 Message-ID: <20150105204628.26559.34214@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org Auto-Submitted: auto-generated Sender: "Christopher Tubbs" X-ReviewGroup: accumulo X-ReviewRequest-URL: https://reviews.apache.org/r/29386/ X-Sender: "Christopher Tubbs" References: <20141231212445.31225.83682@reviews.apache.org> In-Reply-To: <20141231212445.31225.83682@reviews.apache.org> Reply-To: "Christopher Tubbs" X-ReviewRequest-Repository: accumulo --===============0091702226384659264== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/29386/#review66719 ----------------------------------------------------------- I haven't finished looking at all the most recent changes, but here's two quick ones. core/src/main/java/org/apache/accumulo/core/client/security/tokens/KerberosToken.java No such @param server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosAuthenticator.java This makes the assumption that this method is only called on the current user. That's not true. It also supports the public API SecurityOperations.authenticateUser() - Christopher Tubbs On Dec. 31, 2014, 4:24 p.m., Josh Elser wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/29386/ > ----------------------------------------------------------- > > (Updated Dec. 31, 2014, 4:24 p.m.) > > > Review request for accumulo. > > > Bugs: ACCUMULO-2815 > https://issues.apache.org/jira/browse/ACCUMULO-2815 > > > Repository: accumulo > > > Description > ------- > > ACCUMULO-2815 Initial support for Kerberos client authentication. > > Leverage SASL transport provided by Thrift which can speak GSSAPI, which Kerberos implements. Introduced... > > * An Accumulo KerberosToken which is an AuthenticationToken to validate users. > * Custom thrift processor and invocation handler to ensure server RPCs have a valid KRB identity and Accumulo authentication. > * A KerberosAuthenticator which extends ZKAuthenticator to support Kerberos identities seamlessly. > * New ClientConf variables to use SASL transport and pass Kerberos server principal > * Updated ClientOpts and Shell opts to transparently use a KerberosToken when SASL is enabled (no extra client work). > > I believe this is the "bare minimum" for Kerberos support. They are also grossly lacking in unit and integration tests. I believe that I might have somehow broken the client address string in the server (I saw log messages with client: null, but I'm not sure if it's due to these changes or not). A necessary limitation in the Thrift server used is that, like the SSL transport, the SASL transport cannot presently be used with the TFramedTransport, which means none of the [half]async thrift servers will function with this -- we're stuck with the TThreadPoolServer. > > Performed some contrived benchmarks on my laptop (while still using it myself) to get at big-picture view of the performance impact against "normal" operation and Kerberos alone. Each "run" was the duration to ingest 100M records using continuous-ingest, timed with `time`, using 'real'. > > THsHaServer (our default), 6 runs: > > Avg: 10m7.273s (607.273s) > Min: 9m43.395s > Max: 10m52.715s > > TThreadPoolServer (no SASL), 5 runs: > > Avg: 11m16.254s (676.254s) > Min: 10m30.987s > Max: 12m24.192s > > TThreadPoolServer+SASL/GSSAPI (these changes), 6 runs: > > Avg: 13m17.187s (797.187s) > Min: 10m52.997s > Max: 16m0.975s > > The general takeway is that there's about 15% performance degredation in its initial state which is in the realm of what I expected (~10%). > > > Diffs > ----- > > core/src/main/java/org/apache/accumulo/core/cli/ClientOpts.java f6ea934 > core/src/main/java/org/apache/accumulo/core/client/ClientConfiguration.java 6fe61a5 > core/src/main/java/org/apache/accumulo/core/client/impl/ClientContext.java e75bec6 > core/src/main/java/org/apache/accumulo/core/client/impl/ConnectorImpl.java f481cc3 > core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportKey.java 6dc846f > core/src/main/java/org/apache/accumulo/core/client/impl/ThriftTransportPool.java 5da803b > core/src/main/java/org/apache/accumulo/core/client/security/tokens/KerberosToken.java PRE-CREATION > core/src/main/java/org/apache/accumulo/core/conf/Property.java e054a5f > core/src/main/java/org/apache/accumulo/core/rpc/FilterTransport.java PRE-CREATION > core/src/main/java/org/apache/accumulo/core/rpc/SaslConnectionParams.java PRE-CREATION > core/src/main/java/org/apache/accumulo/core/rpc/TTimeoutTransport.java 6eace77 > core/src/main/java/org/apache/accumulo/core/rpc/ThriftUtil.java 09bd6c4 > core/src/main/java/org/apache/accumulo/core/rpc/UGIAssumingTransport.java PRE-CREATION > core/src/main/java/org/apache/accumulo/core/rpc/UGIAssumingTransportFactory.java PRE-CREATION > core/src/main/java/org/apache/accumulo/core/security/Credentials.java 525a958 > core/src/test/java/org/apache/accumulo/core/cli/TestClientOpts.java ff49bc0 > core/src/test/java/org/apache/accumulo/core/client/ClientConfigurationTest.java PRE-CREATION > core/src/test/java/org/apache/accumulo/core/conf/ClientConfigurationTest.java 40be70f > core/src/test/java/org/apache/accumulo/core/rpc/SaslConnectionParamsTest.java PRE-CREATION > proxy/src/main/java/org/apache/accumulo/proxy/Proxy.java 4b048eb > server/base/src/main/java/org/apache/accumulo/server/AccumuloServerContext.java 09ae4f4 > server/base/src/main/java/org/apache/accumulo/server/init/Initialize.java 046cfb5 > server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingInvocationHandler.java PRE-CREATION > server/base/src/main/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingWrapper.java PRE-CREATION > server/base/src/main/java/org/apache/accumulo/server/rpc/TServerUtils.java 641c0bf > server/base/src/main/java/org/apache/accumulo/server/rpc/ThriftServerType.java PRE-CREATION > server/base/src/main/java/org/apache/accumulo/server/security/SecurityOperation.java 5e81018 > server/base/src/main/java/org/apache/accumulo/server/security/SecurityUtil.java 29e4939 > server/base/src/main/java/org/apache/accumulo/server/security/SystemCredentials.java a59d57c > server/base/src/main/java/org/apache/accumulo/server/security/handler/KerberosAuthenticator.java PRE-CREATION > server/base/src/main/java/org/apache/accumulo/server/thrift/UGIAssumingProcessor.java PRE-CREATION > server/base/src/test/java/org/apache/accumulo/server/AccumuloServerContextTest.java PRE-CREATION > server/base/src/test/java/org/apache/accumulo/server/rpc/TCredentialsUpdatingInvocationHandlerTest.java PRE-CREATION > server/base/src/test/java/org/apache/accumulo/server/security/SystemCredentialsTest.java 4202a7e > server/gc/src/main/java/org/apache/accumulo/gc/SimpleGarbageCollector.java 93a9a49 > server/gc/src/test/java/org/apache/accumulo/gc/GarbageCollectWriteAheadLogsTest.java f98721f > server/gc/src/test/java/org/apache/accumulo/gc/SimpleGarbageCollectorTest.java 99558b8 > server/gc/src/test/java/org/apache/accumulo/gc/replication/CloseWriteAheadLogReferencesTest.java cad1e01 > server/master/src/main/java/org/apache/accumulo/master/Master.java 12195fa > server/tracer/src/main/java/org/apache/accumulo/tracer/TraceServer.java 7e33300 > server/tserver/src/main/java/org/apache/accumulo/tserver/TabletServer.java d5c1d2f > shell/src/main/java/org/apache/accumulo/shell/Shell.java 58308ff > shell/src/main/java/org/apache/accumulo/shell/ShellOptionsJC.java 8167ef8 > shell/src/test/java/org/apache/accumulo/shell/ShellConfigTest.java 0e72c8c > shell/src/test/java/org/apache/accumulo/shell/ShellOptionsJCTest.java PRE-CREATION > test/src/main/java/org/apache/accumulo/test/functional/ZombieTServer.java eb84533 > test/src/main/java/org/apache/accumulo/test/performance/thrift/NullTserver.java 2ebc2e3 > test/src/test/java/org/apache/accumulo/server/security/SystemCredentialsIT.java fb71f5f > > Diff: https://reviews.apache.org/r/29386/diff/ > > > Testing > ------- > > Ensure existing unit tests still function. Accumulo is functional and ran continuous ingest multiple times using a client with only a Kerberos identity (no user/password provided). Used MIT Kerberos with Apache Hadoop 2.6.0 and Apache ZooKeeper 3.4.5. > > > Thanks, > > Josh Elser > > --===============0091702226384659264==--