hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-7558) HCatLoader reuses credentials across jobs
Date Fri, 01 Aug 2014 22:37:39 GMT

     [ https://issues.apache.org/jira/browse/HIVE-7558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Dai resolved HIVE-7558.
------------------------------

      Resolution: Fixed
    Hadoop Flags: Reviewed

Patch committed to trunk. Thanks Thiruvel!

> HCatLoader reuses credentials across jobs
> -----------------------------------------
>
>                 Key: HIVE-7558
>                 URL: https://issues.apache.org/jira/browse/HIVE-7558
>             Project: Hive
>          Issue Type: Bug
>          Components: HCatalog
>    Affects Versions: 0.13.1
>            Reporter: Thiruvel Thirumoolan
>            Assignee: Thiruvel Thirumoolan
>             Fix For: 0.14.0
>
>         Attachments: HIVE-7558.patch
>
>
> HCatLoader reuses credentials of stage1 in stage2 for some of the pig queries. This causes
stage-2 to fail, if stage-2 runs for more than 10 mins. Pig queries which loads data using
HCatLoader, filters only by partition columns and does an order by will run into this problem.
Exceptions will be very similar to the following:
> 2014-07-22 17:28:49,337 [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2997:
Unable to recreate exception from backed error: AttemptID:<attemptid> Info:RemoteTrace:

> org.apache.hadoop.security.token.SecretManager$InvalidToken: token (HDFS_DELEGATION_TOKEN
token <tokenid> for <user>) can't be found in cache
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1095)
> 	at org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:195)
> 	at $Proxy7.getFileInfo(Unknown Source)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:102)
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:67)
> 	at $Proxy7.getFileInfo(Unknown Source)
> 	at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1305)
> 	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:734)
> 	at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:176)
> 	at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:51)
> 	at org.apache.hadoop.yarn.util.FSDownload$1.run(FSDownload.java:284)
> 	at org.apache.hadoop.yarn.util.FSDownload$1.run(FSDownload.java:282)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1300)
> 	at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:281)
> 	at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:51)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> 	at java.lang.Thread.run(Thread.java:722)
>  at LocalTrace: 
> 	org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl: token (HDFS_DELEGATION_TOKEN
token <tokenid> for <user>) can't be found in cache
> 	at org.apache.hadoop.yarn.server.nodemanager.api.protocolrecords.impl.pb.LocalResourceStatusPBImpl.convertFromProtoFormat(LocalResourceStatusPBImpl.java:217)
> 	at org.apache.hadoop.yarn.server.nodemanager.api.protocolrecords.impl.pb.LocalResourceStatusPBImpl.getException(LocalResourceStatusPBImpl.java:147)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.update(ResourceLocalizationService.java:823)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.processHeartbeat(ResourceLocalizationService.java:497)
> 	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.heartbeat(ResourceLocalizationService.java:224)
> 	at org.apache.hadoop.yarn.server.nodemanager.api.impl.pb.service.LocalizationProtocolPBServiceImpl.heartbeat(LocalizationProtocolPBServiceImpl.java:46)
> 	at org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2.callBlockingMethod(LocalizationProtocol.java:57)
> 	at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Server.call(ProtoOverHadoopRpcEngine.java:353)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1476)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1472)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1300)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1470)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message