pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohini Palaniswamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2940) HBaseStorage store fails in secure cluster
Date Tue, 02 Oct 2012 04:47:07 GMT

    [ https://issues.apache.org/jira/browse/PIG-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467481#comment-13467481

Rohini Palaniswamy commented on PIG-2940:

   Are you sure it tries to get delegation token in the backend? Because the first check in
the method is if it is front end.

private void addHBaseDelegationToken(Configuration hbaseConf, Job job) {
 	if (!UDFContext.getUDFContext().isFrontend()) {
        if ("kerberos".equalsIgnoreCase(hbaseConf.get(HBASE_SECURITY_CONF_KEY))) {

> HBaseStorage store fails in secure cluster
> ------------------------------------------
>                 Key: PIG-2940
>                 URL: https://issues.apache.org/jira/browse/PIG-2940
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>              Labels: hbase
>             Fix For: 0.11
>         Attachments: PIG-2940.patch
> To reproduce ths issue, please do the following in secure hadoop/hbase cluster:
> # On a gateway node, run kinit to obtain kerberos credentials and run a Pig script that
includes a HBaseStorage load/store.
> # In the front-end, HBaseStorage obtains a delegation token from hbase server and adds
it to the JobConf object.
> # In the back-end, mappers connect to hbase using the delegation token w/o kerberos credentials.
> While load-from-hbase works perfectly fine, store-to-hbase fails. This is because at
step 3, mappers attempt to obtain a delegation token from hbase in the back-end.
> {code:title=setStoreLocation()}
> // Not setting a udf property and getting the hbase delegation token
> // only once like in setLocation as setStoreLocation gets different Job
> // objects for each call and the last Job passed is the one that is
> // launched. So we end up getting multiple hbase delegation tokens.
> addHBaseDelegationToken(m_conf, job);
> {code}
> The problem is that mappers in the back-end don't have kerberos credentials, so the call
to addHBaseDelegationToken() fails with the following error:
> {code}
> 2012-09-30 14:33:42,310 ERROR [main] org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:testuser (auth:SIMPLE) cause:org.apache.hadoop.hbase.security.AccessDeniedException:
org.apache.hadoop.hbase.security.AccessDeniedException: Token generation only allowed for
Kerberos authenticated clients
> 	at org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)
> {code}
> This is not an issue with load because a delegation token is only obtained in the front-end
for the first time when HBASE_TOKEN_SET is not set.
> {code:title=setLocation()}
> String delegationTokenSet = udfProps.getProperty(HBASE_TOKEN_SET);
> if (delegationTokenSet == null) {
>     addHBaseDelegationToken(m_conf, job);
>     udfProps.setProperty(HBASE_TOKEN_SET, "true");
> }
> {code}
> The proposed fix is to modify addHBaseDelegationToken() so that tokens are obtained only
if the current user has kerberos credentials, which is true in the front-end while false in
the back-end.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message