From user-return-26497-archive-asf-public=cust-asf.ponee.io@hive.apache.org  Sun Dec 23 02:11:56 2018
Return-Path: <user-return-26497-archive-asf-public=cust-asf.ponee.io@hive.apache.org>
X-Original-To: archive-asf-public@cust-asf.ponee.io
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by mx-eu-01.ponee.io (Postfix) with SMTP id 2B106180671
	for <archive-asf-public@cust-asf.ponee.io>; Sun, 23 Dec 2018 02:11:56 +0100 (CET)
Received: (qmail 2331 invoked by uid 500); 23 Dec 2018 01:11:54 -0000
Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hive.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hive.apache.org>
List-Post: <mailto:user@hive.apache.org>
List-Id: <user.hive.apache.org>
Reply-To: user@hive.apache.org
Delivered-To: mailing list user@hive.apache.org
Received: (qmail 2322 invoked by uid 99); 23 Dec 2018 01:11:54 -0000
Received: from mail-relay.apache.org (HELO mailrelay1-lw-us.apache.org) (207.244.88.152)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 23 Dec 2018 01:11:54 +0000
Received: from [192.168.2.104] (76-210-4-51.lightspeed.sntcca.sbcglobal.net [76.210.4.51])
	by mailrelay1-lw-us.apache.org (ASF Mail Server at mailrelay1-lw-us.apache.org) with ESMTPSA id CAEBA4301
	for <user@hive.apache.org>; Sun, 23 Dec 2018 01:11:53 +0000 (UTC)
User-Agent: Microsoft-MacOutlook/10.13.0.181109
Date: Sat, 22 Dec 2018 17:11:30 -0800
Subject: Re: hive 3.1 mapjoin with complex predicate produce incorrect results
From: Gopal Vijayaraghavan <gopalv@apache.org>
To: "user@hive.apache.org" <user@hive.apache.org>
Message-ID: <53CD293B-7375-42D3-AAEC-AF19ECFFFAF5@hortonworks.com>
Thread-Topic: hive 3.1 mapjoin with complex predicate produce incorrect
 results
References: <CAAsqibwpeRAjRo-XOKFoKGTi8M9OY=H_Hv6B-AOYsfBXrsuWow@mail.gmail.com>
 <A9B9C317-9115-498E-895E-011B54F0511A@hortonworks.com>
 <CAAsqibzQsSxDfr7NMgMdBb6DXDDbtbpx=DLqE_j8uHrc6-xjbg@mail.gmail.com>
In-Reply-To: <CAAsqibzQsSxDfr7NMgMdBb6DXDDbtbpx=DLqE_j8uHrc6-xjbg@mail.gmail.com>
Mime-version: 1.0
Content-type: text/plain;
	charset="UTF-8"
Content-transfer-encoding: quoted-printable

Hi,

> Subject: Re: hive 3.1 mapjoin with complex predicate produce incorrect re=
sults
...
> |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00 if(_col0 is null, 44, _col0) (ty=
pe: int) |
> |=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A01 _col0 (type: int)=C2=A0 =C2=A0 =C2=A0 =C2=A0 |

That rewrite is pretty neat, but I feel like the IF expression nesting is w=
hat is broken here.

Can you run the same query with "set hive.vectorized.reuse.scratch.columns=3D=
false;" and see if this is a join expression column reuse problem.

If that does work, can you send out a=20

explain vectorization detail <query>;

I'll eventually get back to my dev env in a week, but this looks like a low=
-level exec issue right now.

Cheers,
Gopal