pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Santhosh Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-2940) HBaseStorage store fails in secure cluster
Date Mon, 22 Oct 2012 06:56:14 GMT

    [ https://issues.apache.org/jira/browse/PIG-2940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13481222#comment-13481222
] 

Santhosh Srinivasan commented on PIG-2940:
------------------------------------------

[~rohini] and [~daijy]: while the symptom was exhibited with HBaseStorage on a secure cluster,
the actual fix was in the code that determined if the execution environment was the front
end or the back end. IMHO, that part of the code should be unit tested independent of the
HBaseStorage (secure or otherwise).
                
> HBaseStorage store fails in secure cluster
> ------------------------------------------
>
>                 Key: PIG-2940
>                 URL: https://issues.apache.org/jira/browse/PIG-2940
>             Project: Pig
>          Issue Type: Bug
>            Reporter: Cheolsoo Park
>            Assignee: Cheolsoo Park
>              Labels: hbase
>             Fix For: 0.11, 0.10.1
>
>         Attachments: container_log, PIG-2940-2.patch, PIG-2940.patch
>
>
> To reproduce ths issue, please do the following in secure hadoop/hbase cluster:
> # On a gateway node, run kinit to obtain kerberos credentials and run a Pig script that
includes a HBaseStorage load/store.
> # In the front-end, HBaseStorage obtains a delegation token from hbase server and adds
it to the JobConf object.
> # In the back-end, mappers connect to hbase using the delegation token w/o kerberos credentials.
> While load-from-hbase works perfectly fine, store-to-hbase fails. This is because at
step 3, mappers attempt to obtain a delegation token from hbase in the back-end.
> {code:title=setStoreLocation()}
> // Not setting a udf property and getting the hbase delegation token
> // only once like in setLocation as setStoreLocation gets different Job
> // objects for each call and the last Job passed is the one that is
> // launched. So we end up getting multiple hbase delegation tokens.
> addHBaseDelegationToken(m_conf, job);
> {code}
> The problem is that mappers in the back-end don't have kerberos credentials, so the call
to addHBaseDelegationToken() fails with the following error:
> {code}
> 2012-09-30 14:33:42,310 ERROR [main] org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:testuser (auth:SIMPLE) cause:org.apache.hadoop.hbase.security.AccessDeniedException:
org.apache.hadoop.hbase.security.AccessDeniedException: Token generation only allowed for
Kerberos authenticated clients
> 	at org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:87)
> {code}
> This is not an issue with load because a delegation token is only obtained in the front-end
for the first time when HBASE_TOKEN_SET is not set.
> {code:title=setLocation()}
> String delegationTokenSet = udfProps.getProperty(HBASE_TOKEN_SET);
> if (delegationTokenSet == null) {
>     addHBaseDelegationToken(m_conf, job);
>     udfProps.setProperty(HBASE_TOKEN_SET, "true");
> }
> {code}
> The proposed fix is to modify addHBaseDelegationToken() so that tokens are obtained only
if the current user has kerberos credentials, which is true in the front-end while false in
the back-end.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message