Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1CF711151B for ; Mon, 23 Jun 2014 03:52:27 +0000 (UTC) Received: (qmail 67499 invoked by uid 500); 23 Jun 2014 03:52:26 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 67315 invoked by uid 500); 23 Jun 2014 03:52:26 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 66948 invoked by uid 500); 23 Jun 2014 03:52:26 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 66945 invoked by uid 99); 23 Jun 2014 03:52:26 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 23 Jun 2014 03:52:26 +0000 Date: Mon, 23 Jun 2014 03:52:26 +0000 (UTC) From: "George Wong (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-7273) Hive job fails due to unable to rename reducer output file MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-7273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040392#comment-14040392 ] George Wong commented on HIVE-7273: ----------------------------------- I went through the code of FileSink operator The code is like this. {code:java} if ((bDynParts || isSkewedStoredAsSubDirectories) && !fs.exists(finalPaths[idx].getParent())) { fs.mkdirs(finalPaths[idx].getParent()); } {code} I am wondering why we should check bDynParts and isSkewedStoredAsSubDirectories. In the code, the output is move to finalPath no matter what the values of bDynParts and isSkewedStoredAsSubDirectories are. Since the date move is not avoidable, why not change the code to the following to make sure the path exists before moving the file. {code:java} if (!fs.exists(finalPaths[idx].getParent())) { fs.mkdirs(finalPaths[idx].getParent()); } {code} > Hive job fails due to unable to rename reducer output file > ---------------------------------------------------------- > > Key: HIVE-7273 > URL: https://issues.apache.org/jira/browse/HIVE-7273 > Project: Hive > Issue Type: Bug > Components: Query Processor > Affects Versions: 0.12.0 > Reporter: George Wong > > We ran into this issue in our cluster. > The error message is like this > {noformat} > org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename output from: hdfs://***:8020/tmp/hive-svcckppi/hive_2014-06-16_20-24-09_584_6615934756634587679/_task_tmp.-ext-10002/_tmp.000000_3 to: hdfs://***:8020/tmp/hive-svcckppi/hive_2014-06-16_20-24-09_584_6615934756634587679/_tmp.-ext-10002/000000_3 > at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.commit(FileSinkOperator.java:197) > at org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.access$300(FileSinkOperator.java:108) > at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:867) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:588) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:597) > at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:597) > at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:309) > at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:470) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:407) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:396) > at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1478) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153) > {noformat} > The log of NameNode shows > {noformat} > 2014-06-16 20:43:38,582 WARN org.apache.hadoop.hdfs.StateChange: DIR* FSDirectory.unprotectedRenameTo: failed to rename /tmp/hive-svcckppi/hive_2014-06-16_20-24-09_584_6615934756634587679/_task_tmp.-ext-10002/_tmp.000000_3 to /tmp/hive-svcckppi/hive_2014-06-16_20-24-09_584_6615934756634587679/_tmp.-ext-10002/000000_3 because destination's parent does not exist > {noformat} -- This message was sent by Atlassian JIRA (v6.2#6252)