hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhangduo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12953) RegionServer is not functionally working with AysncRpcClient in secure mode
Date Sat, 21 Feb 2015 12:55:11 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14330210#comment-14330210
] 

zhangduo commented on HBASE-12953:
----------------------------------

Add some ugly logs in DefaultVisibilityLabelServiceImpl and VisibilityLabelsCache.
{code:title=DefaultVisibilityLabelServiceImpl.java}
  public OperationStatus[] addLabels(List<byte[]> labels) throws IOException {
    ...
    if (mutateLabelsRegion(puts, finalOpStatus)) {
      updateZk(true);
    }
    for (byte[] label : labels) {
      String labelStr = Bytes.toString(label);
      LOG.info(labelStr + "=====" + this.labelsCache.getLabelOrdinal(labelStr));
    }
    ...
  }
{code}

{code:title=VisibilityLabelsCache.java}
  public void refreshLabelsCache(byte[] data) throws IOException {
    LOG.info("========refresh", new Exception());
    ...
  }
{code}

And I collected the log for setupBeforeClass
{noformat}
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
secret=====0
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
topsecret=====0
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
confidential=====0
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
public=====0
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
private=====0
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
©ABC=====0
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
ु=====0
2015-02-21 20:39:16,362 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
©ABC"ु\secret"'&\=====0
2015-02-21 20:39:16,363 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
'"+=====0
2015-02-21 20:39:16,363 INFO  [B.defaultRpcServer.handler=0,queue=0,port=42678] visibility.DefaultVisibilityLabelServiceImpl(236):
-?=====0
2015-02-21 20:39:16,363 INFO  [main-EventThread] visibility.VisibilityLabelsCache(115): ========refresh
java.lang.Exception
	at org.apache.hadoop.hbase.security.visibility.VisibilityLabelsCache.refreshLabelsCache(VisibilityLabelsCache.java:115)
	at org.apache.hadoop.hbase.security.visibility.ZKVisibilityLabelWatcher.refreshVisibilityLabelsCache(ZKVisibilityLabelWatcher.java:76)
	at org.apache.hadoop.hbase.security.visibility.ZKVisibilityLabelWatcher.nodeDataChanged(ZKVisibilityLabelWatcher.java:115)
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:412)
	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
{noformat}

We can see that, refresh is triggered from a background zk watcher, and it is possible that
the labels are not in labelsCache after addLabels being called.

I think this is the reason why TestShell is flakey, but I do not know what is the right way
to fix it(make test wait or modify addLabels?). [~stack]

> RegionServer is not functionally working with AysncRpcClient in secure mode
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-12953
>                 URL: https://issues.apache.org/jira/browse/HBASE-12953
>             Project: HBase
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.0.0, 1.1.0
>            Reporter: Ashish Singhi
>            Assignee: zhangduo
>            Priority: Critical
>             Fix For: 2.0.0, 1.1.0
>
>         Attachments: HBASE-12953.patch, HBASE-12953_1.patch, HBASE-12953_2.patch, HBASE-12953_2.patch,
HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_2.patch, HBASE-12953_3 (2).patch, HBASE-12953_3
(2).patch, HBASE-12953_3.patch, HBASE-12953_3.patch, HBASE-12953_3.patch, testcase.patch
>
>
> HBase version 2.0.0
> Default value for {{hbase.rpc.client.impl}} is set to AsyncRpcClient.
> When trying to install HBase with Kerberos, RegionServer is not working functionally.
> The following log is logged in its log file
> {noformat}
> 2015-02-02 14:59:05,407 WARN  [AsyncRpcChannel-pool1-t1] channel.DefaultChannelPipeline:
An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually
means the last handler in the pipeline did not handle the exception.
> io.netty.channel.ChannelPipelineException: org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded()
has thrown an exception; removed.
> 	at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:499)
> 	at io.netty.channel.DefaultChannelPipeline.callHandlerAdded(DefaultChannelPipeline.java:481)
> 	at io.netty.channel.DefaultChannelPipeline.addFirst0(DefaultChannelPipeline.java:114)
> 	at io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:97)
> 	at io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:235)
> 	at io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:214)
> 	at org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:194)
> 	at org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:157)
> 	at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
> 	at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
> 	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
> 	at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
> 	at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> 	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:253)
> 	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:288)
> 	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
> 	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
> 	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
> 	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
> 	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException:
No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
> 	at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
> 	at org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded(SaslClientHandler.java:154)
> 	at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:486)
> 	... 20 more
> Caused by: GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)
> 	at sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
> 	at sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
> 	at sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> 	at sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
> 	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
> 	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> 	at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
> {noformat}
> When set hbase.rpc.client.impl to RpcClientImpl, there seems to be no issue.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message