Return-Path: X-Original-To: apmail-hive-issues-archive@minotaur.apache.org Delivered-To: apmail-hive-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 859AF189A2 for ; Wed, 3 Jun 2015 17:51:40 +0000 (UTC) Received: (qmail 92952 invoked by uid 500); 3 Jun 2015 17:51:39 -0000 Delivered-To: apmail-hive-issues-archive@hive.apache.org Received: (qmail 92817 invoked by uid 500); 3 Jun 2015 17:51:39 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 92624 invoked by uid 99); 3 Jun 2015 17:51:39 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 03 Jun 2015 17:51:39 +0000 Date: Wed, 3 Jun 2015 17:51:39 +0000 (UTC) From: "Sergey Shelukhin (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-10735) LLAP: Cached plan race condition - VectorMapJoinCommonOperator has no closeOp() MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-10735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571398#comment-14571398 ] Sergey Shelukhin commented on HIVE-10735: ----------------------------------------- Why does this patch remove thread-safe position? This has been known to cause problems in the past when hashtable is shared, which is something that may be done in LLAP > LLAP: Cached plan race condition - VectorMapJoinCommonOperator has no closeOp() > ------------------------------------------------------------------------------- > > Key: HIVE-10735 > URL: https://issues.apache.org/jira/browse/HIVE-10735 > Project: Hive > Issue Type: Sub-task > Components: Vectorization > Reporter: Gopal V > Assignee: Matt McCline > Priority: Critical > Attachments: HIVE-10705.01.patch, HIVE-10705.02.patch > > > Looks like some state is mutated during execution across threads in LLAP. > Either we can't share the operator objects across threads, because they are tied to the data objects per invocation or this is missing a closeOp() which resets the common-setup between reuses. > {code} > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ArrayIndexOutOfBoundsException > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerBigOnlyLongOperator.process(VectorMapJoinInnerBigOnlyLongOperator.java:380) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:850) > at org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.process(VectorFilterOperator.java:114) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:850) > at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97) > at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:164) > at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45) > ... 18 more > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ArrayIndexOutOfBoundsException > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:379) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:850) > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.forwardBigTableBatch(VectorMapJoinGenerateResultOperator.java:599) > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerBigOnlyGenerateResultOperator.generateHashMultiSetResultRepeatedAll(VectorMapJoinInnerBigOnlyGenerateResultOperator.java:304) > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerBigOnlyGenerateResultOperator.finishInnerBigOnlyRepeated(VectorMapJoinInnerBigOnlyGenerateResultOperator.java:328) > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerBigOnlyLongOperator.process(VectorMapJoinInnerBigOnlyLongOperator.java:201) > ... 24 more > Caused by: java.lang.ArrayIndexOutOfBoundsException > at org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:152) > at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow$StringReaderByValue.apply(VectorDeserializeRow.java:349) > at org.apache.hadoop.hive.ql.exec.vector.VectorDeserializeRow.deserializeByValue(VectorDeserializeRow.java:688) > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinGenerateResultOperator.generateHashMapResultSingleValue(VectorMapJoinGenerateResultOperator.java:177) > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerGenerateResultOperator.finishInner(VectorMapJoinInnerGenerateResultOperator.java:201) > at org.apache.hadoop.hive.ql.exec.vector.mapjoin.VectorMapJoinInnerLongOperator.process(VectorMapJoinInnerLongOperator.java:359) > ... 29 more > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)