Return-Path: X-Original-To: apmail-hive-dev-archive@www.apache.org Delivered-To: apmail-hive-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0BEE69140 for ; Thu, 18 Dec 2014 07:25:14 +0000 (UTC) Received: (qmail 31139 invoked by uid 500); 18 Dec 2014 07:25:13 -0000 Delivered-To: apmail-hive-dev-archive@hive.apache.org Received: (qmail 31066 invoked by uid 500); 18 Dec 2014 07:25:13 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 31053 invoked by uid 500); 18 Dec 2014 07:25:13 -0000 Delivered-To: apmail-hadoop-hive-dev@hadoop.apache.org Received: (qmail 31050 invoked by uid 99); 18 Dec 2014 07:25:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 18 Dec 2014 07:25:13 +0000 Date: Thu, 18 Dec 2014 07:25:13 +0000 (UTC) From: "Navis (JIRA)" To: hive-dev@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-9123) Query with join fails with NPE when using join auto conversion MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-9123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14251327#comment-14251327 ] Navis commented on HIVE-9123: ----------------------------- [~kgs] Cannot reproduce in trunk. Could you try with hive-0.14.1? > Query with join fails with NPE when using join auto conversion > -------------------------------------------------------------- > > Key: HIVE-9123 > URL: https://issues.apache.org/jira/browse/HIVE-9123 > Project: Hive > Issue Type: Bug > Affects Versions: 0.13.1 > Environment: CDH5 with Hive 0.13.1 > Reporter: Kamil Gorlo > > I have two simple tables: > desc kgorlo_comm; > | col_name | data_type | comment | > | id | bigint | | > | dest_id | bigint | | > desc kgorlo_log; > | col_name | data_type | comment | > | id | bigint | | > | dest_id | bigint | | > | tstamp | bigint | | > With data: > select * from kgorlo_comm; > | kgorlo_comm.id | kgorlo_comm.dest_id | > | 1 | 2 | > | 2 | 1 | > | 1 | 3 | > | 2 | 3 | > | 3 | 5 | > | 4 | 5 | > select * from kgorlo_log; > | kgorlo_log.id | kgorlo_log.dest_id | kgorlo_log.tstamp | > | 1 | 2 | 0 | > | 1 | 3 | 0 | > | 1 | 5 | 0 | > | 3 | 1 | 0 | > Following query fails in second stage of execution: > bq. select v.id, v.dest_id from kgorlo_log v join (select id, dest_id, count(*) as wiad from kgorlo_comm group by id, dest_id)com1 on com1.id=v.id and com1.dest_id=v.dest_id; > with following exception: > {quote} > 2014-12-16 17:09:17,629 ERROR [uber-SubtaskRunner] org.apache.hadoop.hive.ql.exec.MapJoinOperator: Unxpected exception: null > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.exec.MapJoinOperator.getRefKey(MapJoinOperator.java:198) > at org.apache.hadoop.hive.ql.exec.MapJoinOperator.computeMapJoinKey(MapJoinOperator.java:186) > at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:216) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796) > at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796) > at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:540) > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2014-12-16 17:09:17,659 FATAL [uber-SubtaskRunner] org.apache.hadoop.hive.ql.exec.mr.ExecMapper: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"_col0":1,"_col1":2} > at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550) > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runSubtask(LocalContainerLauncher.java:370) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.runTask(LocalContainerLauncher.java:295) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler.access$200(LocalContainerLauncher.java:181) > at org.apache.hadoop.mapred.LocalContainerLauncher$EventHandler$1.run(LocalContainerLauncher.java:224) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unxpected exception: null > at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:254) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796) > at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796) > at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:540) > ... 13 more > Caused by: java.lang.NullPointerException > at org.apache.hadoop.hive.ql.exec.MapJoinOperator.getRefKey(MapJoinOperator.java:198) > at org.apache.hadoop.hive.ql.exec.MapJoinOperator.computeMapJoinKey(MapJoinOperator.java:186) > at org.apache.hadoop.hive.ql.exec.MapJoinOperator.processOp(MapJoinOperator.java:216) > ... 17 more > {quote} > When I set hive.auto.convert.join=false everything works. > Here are explains with this variable turned off and on: > https://gist.github.com/kgs/20db747c8d81d94ac20e > https://gist.github.com/kgs/63bc1fc148354b98a63e -- This message was sent by Atlassian JIRA (v6.3.4#6332)