Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 645C811FFD for ; Tue, 15 Apr 2014 16:22:35 +0000 (UTC) Received: (qmail 80118 invoked by uid 500); 15 Apr 2014 16:22:29 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 80032 invoked by uid 500); 15 Apr 2014 16:22:28 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 79995 invoked by uid 500); 15 Apr 2014 16:22:26 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 79770 invoked by uid 99); 15 Apr 2014 16:22:20 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 15 Apr 2014 16:22:20 +0000 Date: Tue, 15 Apr 2014 16:22:20 +0000 (UTC) From: "Xuefu Zhang (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-6913) Hive unable to find the hashtable file during complex multi-staged map join MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-6913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13969685#comment-13969685 ] Xuefu Zhang commented on HIVE-6913: ----------------------------------- Nice catch, Brock! The code line you referred to seems off the description that you gave. In addition, s it hard to construct a test case? > Hive unable to find the hashtable file during complex multi-staged map join > --------------------------------------------------------------------------- > > Key: HIVE-6913 > URL: https://issues.apache.org/jira/browse/HIVE-6913 > Project: Hive > Issue Type: Bug > Reporter: Brock Noland > Assignee: Brock Noland > Attachments: HIVE-6913.patch > > > If a query has multiple mapjoins and one of the tables to be mapjoined is empty, the query can result in a "no such file or directory" when looking for the hashtable. > This is because when we generate a dummy hash table, we do not close the TableScan (TS) operator for that table. Additionally, HashTableSinkOperator (HTSO) outputs it's hash tables in the closeOp method. However, when close is called on HTSO we check to ensure that all parents are closed: https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java#L333 > which is not true on this case, because the TS operator for the empty table was never closed. -- This message was sent by Atlassian JIRA (v6.2#6252)