hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-6913) Hive unable to find the hashtable file during complex multi-staged map join
Date Wed, 16 Apr 2014 01:30:19 GMT

    [ https://issues.apache.org/jira/browse/HIVE-6913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13970321#comment-13970321
] 

Brock Noland commented on HIVE-6913:
------------------------------------

Thank you for pointing this out! The line of code is: https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java#L567

I tried for an entire day to get a test for this but could not reproduce outside of the production
environment. I did however, test the fix in the production environment and it worked well.

> Hive unable to find the hashtable file during complex multi-staged map join
> ---------------------------------------------------------------------------
>
>                 Key: HIVE-6913
>                 URL: https://issues.apache.org/jira/browse/HIVE-6913
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Brock Noland
>            Assignee: Brock Noland
>         Attachments: HIVE-6913.patch
>
>
> If a query has multiple mapjoins and one of the tables to be mapjoined is empty, the
query can result in a "no such file or directory" when looking for the hashtable.
> This is because when we generate a dummy hash table, we do not close the TableScan (TS)
operator for that table. Additionally, HashTableSinkOperator (HTSO) outputs it's hash tables
in the closeOp method. However, when close is called on HTSO we check to ensure that all parents
are closed: https://github.com/apache/hive/blob/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java#L333
> which is not true on this case, because the TS operator for the empty table was never
closed.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message