hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14690) Query fail when hive.exec.parallel=true, with conflicting session dir
Date Tue, 11 Oct 2016 23:34:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15566958#comment-15566958
] 

Daniel Dai commented on HIVE-14690:
-----------------------------------

Need to mention there is no way to add unit test. The exception only happens on hdfs. Even
MiniCluster is using local filesystem which will not throw exception.

> Query fail when hive.exec.parallel=true, with conflicting session dir
> ---------------------------------------------------------------------
>
>                 Key: HIVE-14690
>                 URL: https://issues.apache.org/jira/browse/HIVE-14690
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.3.0, 2.1.0
>            Reporter: Daniel Dai
>            Assignee: Daniel Dai
>             Fix For: 1.3.0, 2.2.0
>
>         Attachments: HIVE-14690.1.patch, HIVE-14690.2.patch, HIVE-14690.3.patch
>
>
> This happens when hive.scratchdir.lock=true. Error message:
> {code}
> /hive/scratch/343hdirdp/cab907fc-5e1d-4d69-aa72-d7b442495c7a/inuse.info (inode 19537):
File does not exist. [Lease.  Holder: DFSClient_NONMAPREDUCE_1572639975_1, pendingcreates:
2]
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3430)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3235)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3073)
> 	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3033)
> 	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137)
> 	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1668)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
> 	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535)
> 	at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:74)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message