Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 27 Dec 2012 16:48:12 +0000 (UTC)
From: "James Kinley (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12625341.1356623879316.55309.1356626892547@arcas>
In-Reply-To: <JIRA.12625341.1356623879316@arcas>
References: <JIRA.12625341.1356623879316@arcas>
Subject: [jira] [Commented] (HBASE-7442) HBase remote CopyTable not working
 when security enabled
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-7442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13540031#comment-13540031 ] 

James Kinley commented on HBASE-7442:
-------------------------------------

I noticed that *{{CopyTable#createSubmittableJob}}*, and specifically *{{TableMapReduceUtil#initCredentials}}* was only acquiring a delegation token for the local HBase cluster so I added the following to *{{TableMapReduceUtil#initCredentials}}*:

{code}
String quorumAddress = job.getConfiguration().get(TableOutputFormat.QUORUM_ADDRESS);
if (quorumAddress != null) {
  String[] parts = ZKUtil.transformClusterKey(quorumAddress);          
  Configuration peerConf = HBaseConfiguration.create(job.getConfiguration());
  peerConf.set(HConstants.ZOOKEEPER_QUORUM, parts[0]);
  peerConf.set("hbase.zookeeper.client.port", parts[1]);
  peerConf.set(HConstants.ZOOKEEPER_ZNODE_PARENT, parts[2]);
  User.getCurrent().obtainAuthTokenForJob(peerConf, job);
}
{code}

The user now has the correct tokens:

Cluster 1 ID = 7bb01c84-3c66-49ca-82d2-2f9218f13dad
Cluster 2 ID = d4653a37-83df-4929-b950-e03987c3b4e8

{code}
INFO token.TokenUtil: Obtained token HBASE_AUTH_TOKEN for user kinley/admin@KINLEY.COM on cluster 7bb01c84-3c66-49ca-82d2-2f9218f13dad
INFO token.TokenUtil: Obtained token HBASE_AUTH_TOKEN for user kinley/admin@KINLEY.COM on cluster d4653a37-83df-4929-b950-e03987c3b4e8
{code}

But when initialising the *{{HTable}}* connection in the mapper, the *{{TokenSelector}}* in *{{SecureClient#SecureConnection}}* returns the wrong token from the user's token cache. This is because the token cache is keyed on {{clusterId}}, and despite the token for the local cluster existing in the cache ({{key=7bb01c84-3c66-49ca-82d2-2f9218f13dad}}), the {{clusterId}} is fixed at this point in *{{HBaseClient}}* to the remote cluster's ID ({{key=d4653a37-83df-4929-b950-e03987c3b4e8}}), and it is this ID that is always passed to the *{{TokenSelector}}*.

I can think of a couple of possible solutions:
1. Either pass the {{clusterId}} into *{{SecureClient#SecureConnection}}* as part of the *{{ConnectionId}}*. This way it can be used by the *{{TokenSelector}}* instead of the fixed ID in the parent class *{{HBaseClient}}*?, or 
2. *{{SecureRpcEngine}}* caches multiple clients based on a *{{SocketFactory}}*, so we could maintain separate clients for both the local and remote clusters here?
                
> HBase remote CopyTable not working when security enabled
> --------------------------------------------------------
>
>                 Key: HBASE-7442
>                 URL: https://issues.apache.org/jira/browse/HBASE-7442
>             Project: HBase
>          Issue Type: Bug
>          Components: IPC/RPC, mapreduce, security
>    Affects Versions: 0.92.1
>            Reporter: James Kinley
>         Attachments: attempt_201212271546_0001_m_000000_0.log
>
>
> When security is enabled, HBase CopyTable fails with Kerberos exception:
> {code}
> FATAL org.apache.hadoop.ipc.SecureClient: SASL authentication failed. The most likely cause is missing or invalid credentials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
> {code}
> This is only when copying to remote HBase cluster (using either MRv1 or YARN), local copy works fine.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira