hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laxman <lakshman...@huawei.com>
Subject RE: Secure HBase setup
Date Wed, 23 May 2012 03:59:25 GMT
This issue is resolved after replacing the Java JCE jars on client side as well.
I feel its worth documenting in HBase book.

--
Regards,
Laxman
> -----Original Message-----
> From: Laxman [mailto:lakshman.ch@huawei.com]
> Sent: Tuesday, May 22, 2012 3:21 PM
> To: dev@hbase.apache.org
> Subject: Secure HBase setup
> 
> We got stuck with a problem while verifying client authentication in a
> secure HBase cluster.
> We are able to start a secure HBase cluster successfully.
> 
> However, clients are not able to establish secure connection with HBase
> server successfully.
> 
> Other details:
> HBase version: 0.94.0
> Hadoop version: 0.23.1
> Kerberos version: 1.10.1
> Java version: 1.6.0_31, 64 bit
> Linux version: SuSE 11.1 [Kernel version : 2.6.32.12-0.7-default x86_64
> GNU/Linux]
> 
> We had gone thru the solutions available @
> http://docs.oracle.com/javase/1.5.0/docs/guide/security/jgss/tutorials/
> Troubleshooting.html
> https://ccp.cloudera.com/display/CDHDOC/Appendix+A+-
> +Troubleshooting#AppendixA-Troubleshooting-
> Problem2%3AJavaisunabletoreadtheKerberoscredentialscachecreatedbyversio
> nsofMITKerberos1.8.1orhigher.
> 
> But none of then seems to work. Any clue?
> 
> There are no change in server logs as client is failing is failing even
> before it communicates with server.
> Exception we are hitting (Client side logs):
> 
> 2012-05-22 09:42:22,627 WARN org.apache.hadoop.ipc.SecureClient:
> Exception encountered while connecting to the server :
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to
> find any Kerberos tgt)]
> 2012-05-22 09:42:22,627 ERROR
> org.apache.hadoop.security.UserGroupInformation:
> PriviledgedActionException as:testuser (auth:KERBEROS)
> cause:java.io.IOException: javax.security.sasl.SaslException: GSS
> initiate failed [Caused by GSSException: No valid credentials provided
> (Mechanism level: Failed to find any Kerberos tgt)]
> 2012-05-22 09:42:22,630 DEBUG org.apache.hadoop.ipc.SecureClient:
> closing ipc connection to HOST-10-18-40-19/10.18.40.19:60020:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to
> find any Kerberos tgt)]
> java.io.IOException: javax.security.sasl.SaslException: GSS initiate
> failed [Caused by GSSException: No valid credentials provided
> (Mechanism level: Failed to find any Kerberos tgt)]
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$1.run(SecureC
> lient.java:227)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati
> on.java:1177)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
> va:39)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
> rImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
> 	at org.apache.hadoop.hbase.security.User.call(User.java:586)
> 	at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
> 	at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:
> 440)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.handleSaslCon
> nectionFailure(SecureClient.java:194)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstream
> s(SecureClient.java:274)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.jav
> a:485)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient.getConnection(SecureClient.jav
> a:69)
> 	at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:897)
> 	at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine$Invoker.invoke(SecureRpcEng
> ine.java:164)
> 	at $Proxy6.getProtocolVersion(Unknown Source)
> 	at
> org.apache.hadoop.hbase.ipc.SecureRpcEngine.getProxy(SecureRpcEngine.ja
> va:208)
> 	at
> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
> 	at
> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
> 	at
> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
> 	at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:236)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.getHRegionConnection(HConnectionManager.java:1284)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.getHRegionConnection(HConnectionManager.java:1240)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.getHRegionConnection(HConnectionManager.java:1227)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.locateRegionInMeta(HConnectionManager.java:936)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.locateRegion(HConnectionManager.java:832)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.locateRegion(HConnectionManager.java:801)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.locateRegionInMeta(HConnectionManager.java:933)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.locateRegion(HConnectionManager.java:836)
> 	at
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion.locateRegion(HConnectionManager.java:801)
> 	at
> org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:234)
> 	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:174)
> 	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:133)
> 	at hbase.test.Hbasetest.main(Hbasetest.java:37)
> Caused by: javax.security.sasl.SaslException: GSS initiate failed
> [Caused by GSSException: No valid credentials provided (Mechanism
> level: Failed to find any Kerberos tgt)]
> 	at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Cl
> ient.java:194)
> 	at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSa
> slRpcClient.java:138)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConn
> ection(SecureClient.java:176)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$500(Se
> cureClient.java:84)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureC
> lient.java:267)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureC
> lient.java:264)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:396)
> 	at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati
> on.java:1177)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja
> va:39)
> 	at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso
> rImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.hbase.util.Methods.call(Methods.java:37)
> 	at org.apache.hadoop.hbase.security.User.call(User.java:586)
> 	at org.apache.hadoop.hbase.security.User.access$700(User.java:50)
> 	at
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:
> 440)
> 	at
> org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupIOstream
> s(SecureClient.java:263)
> 	... 23 more
> Caused by: GSSException: No valid credentials provided (Mechanism
> level: Failed to find any Kerberos tgt)
> 	at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredentia
> l.java:130)
> 	at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFac
> tory.java:106)
> 	at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFact
> ory.java:172)
> 	at
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.jav
> a:209)
> 	at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:195
> )
> 	at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:162
> )
> 	at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Cl
> ient.java:175)
> 	... 40 more
> 2012-05-22 09:42:22,636 DEBUG org.apache.hadoop.ipc.SecureClient: IPC
> Client (1778276127) connection to HOST-10-18-40-19/10.18.40.19:60020
> from testuser: closed
> 2012-05-22 09:42:22,638 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion: locateRegionInMeta parentTable=-ROOT-, metaLocation={region=-
> ROOT-,,0.70236052, hostname=HOST-10-18-40-19, port=60020}, attempt=0 of
> 120 failed; retrying after sleep of 1000 because:
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to
> find any Kerberos tgt)]
> 2012-05-22 09:42:22,640 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion: Looked up root region location,
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectio
> nImplementation@6ecf829d; serverName=HOST-10-18-40-
> 19,60020,1337574445438
> 2012-05-22 09:42:23,641 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementa
> tion: Looked up root region location,
> connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectio
> nImplementation@6ecf829d; serverName=HOST-10-18-40-
> 19,60020,1337574445438
> 2012-05-22 09:42:23,642 DEBUG org.apache.hadoop.ipc.SecureClient: RPC
> Server Kerberos principal name for
> protocol=org.apache.hadoop.hbase.ipc.HRegionInterface is
> hbase/hadoop@HADOOP.COM
> 
> 
> --
> Regards,
> Laxman


Mime
View raw message