hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bijieshan <bijies...@huawei.com>
Subject RE: WritableRpcEngine$Invoker caches UGI
Date Mon, 12 May 2014 00:41:49 GMT
Sent again since I got no reply.

That's 1 problem we encountered due to calling User#login more than 1 times. For your reference.

-----Original Message-----
From: Bijieshan 
Sent: Thursday, May 08, 2014 6:54 PM
To: dev@hbase.apache.org
Cc: Fanghao
Subject: WritableRpcEngine$Invoker caches UGI


I think WritableRpcEngine$Invoker should not cache UGI. The intent behind this design?

We encounter 1 problem due to client called User#login more than 1 times. The error log:

14/05/08 15:44:40 FATAL ipc.SecureClient: SASL authentication failed. The most likely cause
is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials
provided (Mechanism level: Failed to find any Kerberos tgt)]
	at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
	at org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:138)
	at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.setupSaslConnection(SecureClient.java:184)
	at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection.access$4(SecureClient.java:180)
	at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:302)
	at org.apache.hadoop.hbase.ipc.SecureClient$SecureConnection$2.run(SecureClient.java:1)
	at java.security.AccessController.doPrivileged(Native Method)

The reason of this exception as below:
(1). Client calls login.
(2). Calls Scan. So client side caches the proxy instance for HRegionInterface, and also caches
the UGI.
(3). Client calls login again(It should not happen, but none protections we have).
(4). Wait until the TGT expire.
(5). Calls Scan again. So it will use the cached old UGI to create the SecureConnection.
(6). Gets a SaslException due to the invalid credential. So triggers SecureClient# handleSaslConnectionFailure
(7). Checks whether need to re-login by SecureClient# shouldAuthenticateOverKrb:
    private synchronized boolean shouldAuthenticateOverKrb() throws IOException {
      UserGroupInformation loginUser = UserGroupInformation.getLoginUser();
      UserGroupInformation currentUser = UserGroupInformation.getCurrentUser();
      UserGroupInformation realUser = currentUser.getRealUser();
      return relogin = authMethod == AuthMethod.KERBEROS && loginUser != null
          // Make sure user logged in using Kerberos either keytab or TGT
          loginUser.hasKerberosCredentials() &&
          // relogin only in case it is the login user (e.g. JT)
          // or superuser (like oozie).
          (loginUser.equals(currentUser) || loginUser.equals(realUser));
The currentUser is the cached old one, but the login user is the new one created by step (3).
So this check returns false and causes the re-login skipped.

The solution of this problem includes:

(1). Provide suggestion that the client side should only login once.
(2). Modify the logic in WritableRpcEngine$Invoker that does not cache UGI.  Does this make

View raw message