Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4CDDE18CAE for ; Tue, 22 Dec 2015 11:40:13 +0000 (UTC) Received: (qmail 4567 invoked by uid 500); 22 Dec 2015 11:40:11 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 4491 invoked by uid 500); 22 Dec 2015 11:40:11 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 4481 invoked by uid 99); 22 Dec 2015 11:40:11 -0000 Received: from Unknown (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 22 Dec 2015 11:40:11 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id B1989C0EEC for ; Tue, 22 Dec 2015 11:40:10 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 4.716 X-Spam-Level: **** X-Spam-Status: No, score=4.716 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=3, MANY_SPAN_IN_TEXT=1.836, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-us-east.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id titMj3bPTCVZ for ; Tue, 22 Dec 2015 11:40:01 +0000 (UTC) Received: from mail-wm0-f50.google.com (mail-wm0-f50.google.com [74.125.82.50]) by mx1-us-east.apache.org (ASF Mail Server at mx1-us-east.apache.org) with ESMTPS id 3D97242989 for ; Tue, 22 Dec 2015 11:40:01 +0000 (UTC) Received: by mail-wm0-f50.google.com with SMTP id p187so106884421wmp.0 for ; Tue, 22 Dec 2015 03:40:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=wU7RLUFkeDOL/PP6ajo5YkC42JK4yrZKF0lNJD+oaQA=; b=DEiP5d6RLqEG4mVBtAbtdpQG12gRyoWF1RAK/SP86tLlR67VG128JoDaL4lcm0oyHU 5hRI4Pkc1mv3muMJbbJUhEnggpWpssxw6ArDX3SKRBjUi0OHbpfAq8FBYJ36y36GvAwb j+U5MZNNhpmnX0yxrDGpzkiTd6NSBc4OfL5I9M5VOjj91eQ53PznIWmkM2Ky7ix4XXso 6/+mbuE5CQ2LufrL43rB7BlSjUpTxvJ6GTdAOv7II/0DUNTp5Ceh4AP03/CZ33v7vCUt tl1VPJ9btOTiO1+eq2cSSoIKkEq4T/n7gGHrUlz7e9Pvvd5wWE4kj3OuPSpZBll+ukpk G6aw== MIME-Version: 1.0 X-Received: by 10.194.7.100 with SMTP id i4mr19130703wja.76.1450784394389; Tue, 22 Dec 2015 03:39:54 -0800 (PST) Received: by 10.194.87.130 with HTTP; Tue, 22 Dec 2015 03:39:54 -0800 (PST) In-Reply-To: References: Date: Tue, 22 Dec 2015 19:39:54 +0800 Message-ID: Subject: Re: It seems that result of Hive on Spark is mistake And result of Hive and Hive on Spark are not the same From: Jone Zhang To: user@hive.apache.org, Xuefu Zhang Content-Type: multipart/alternative; boundary=047d7b5d49de7935b105277b10f2 --047d7b5d49de7935b105277b10f2 Content-Type: text/plain; charset=UTF-8 Hive 1.2.1 on Spark1.4.1 2015-12-22 19:31 GMT+08:00 Jone Zhang : > *select * from staff;* > 1 jone 22 1 > 2 lucy 21 1 > 3 hmm 22 2 > 4 james 24 3 > 5 xiaoliu 23 3 > > *select id,date_ from trade union all select id,"test" from trade ;* > 1 201510210908 > 2 201509080234 > 2 201509080235 > 1 test > 2 test > 2 test > > *set hive.execution.engine=spark;* > *set spark.master=local;* > *select /*+mapjoin(t)*/ * from staff s join * > *(select id,date_ from trade union all select id,"test" from trade ) t on > s.id =t.id ;* > 1 jone 22 1 1 201510210908 > 2 lucy 21 1 2 201509080234 > 2 lucy 21 1 2 201509080235 > > *set hive.execution.engine=mr;* > *select /*+mapjoin(t)*/ * from staff s join * > *(select id,date_ from trade union all select id,"test" from trade ) t on > s.id =t.id ;* > FAILED: SemanticException [Error 10227]: Not all clauses are supported > with mapjoin hint. Please remove mapjoin hint. > > *I have two questions* > *1.Why result of hive on spark not include the following record?* > 1 jone 22 1 1 test > 2 lucy 21 1 2 test > 2 lucy 21 1 2 test > > *2.Why there are two different ways of dealing same query?* > > > *explain 1:* > *set hive.execution.engine=spark;* > *set spark.master=local;* > *explain * > *select id,date_ from trade union all select id,"test" from trade;* > OK > STAGE DEPENDENCIES: > Stage-1 is a root stage > Stage-0 depends on stages: Stage-1 > > STAGE PLANS: > Stage: Stage-1 > Spark > DagName: > jonezhang_20151222191643_5301d90a-caf0-4934-8092-d165c87a4190:1 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: trade > Statistics: Num rows: 6 Data size: 48 Basic stats: > COMPLETE Column stats: NONE > Select Operator > expressions: id (type: int), date_ (type: string) > outputColumnNames: _col0, _col1 > Statistics: Num rows: 6 Data size: 48 Basic stats: > COMPLETE Column stats: NONE > File Output Operator > compressed: false > Statistics: Num rows: 12 Data size: 96 Basic stats: > COMPLETE Column stats: NONE > table: > input format: > org.apache.hadoop.mapred.TextInputFormat > output format: > org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat > serde: > org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe > Map 2 > Map Operator Tree: > TableScan > alias: trade > Statistics: Num rows: 6 Data size: 48 Basic stats: > COMPLETE Column stats: NONE > Select Operator > expressions: id (type: int), 'test' (type: string) > outputColumnNames: _col0, _col1 > Statistics: Num rows: 6 Data size: 48 Basic stats: > COMPLETE Column stats: NONE > File Output Operator > compressed: false > Statistics: Num rows: 12 Data size: 96 Basic stats: > COMPLETE Column stats: NONE > table: > input format: > org.apache.hadoop.mapred.TextInputFormat > output format: > org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat > serde: > org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe > > Stage: Stage-0 > Fetch Operator > limit: -1 > Processor Tree: > ListSink > > > *explain 2:* > *set hive.execution.engine=spark;* > *set spark.master=local;* > *explain * > *select /*+mapjoin(t)*/ * from staff s join * > *(select id,date_ from trade union all select id,"20999999999" from trade > ) t on s.id =t.id ;* > OK > STAGE DEPENDENCIES: > Stage-2 is a root stage > Stage-1 depends on stages: Stage-2 > Stage-0 depends on stages: Stage-1 > > STAGE PLANS: > Stage: Stage-2 > Spark > DagName: > jonezhang_20151222191716_be7eac84-b5b6-4478-b88f-9f59e2b1b1a8:3 > Vertices: > Map 1 > Map Operator Tree: > TableScan > alias: trade > Statistics: Num rows: 6 Data size: 48 Basic stats: > COMPLETE Column stats: NONE > Filter Operator > predicate: id is not null (type: boolean) > Statistics: Num rows: 3 Data size: 24 Basic stats: > COMPLETE Column stats: NONE > Select Operator > expressions: id (type: int), date_ (type: string) > outputColumnNames: _col0, _col1 > Statistics: Num rows: 3 Data size: 24 Basic stats: > COMPLETE Column stats: NONE > Spark HashTable Sink Operator > keys: > 0 id (type: int) > 1 _col0 (type: int) > Local Work: > Map Reduce Local Work > > Stage: Stage-1 > Spark > DagName: > jonezhang_20151222191716_be7eac84-b5b6-4478-b88f-9f59e2b1b1a8:2 > Vertices: > Map 2 > Map Operator Tree: > TableScan > alias: s > Statistics: Num rows: 1 Data size: 66 Basic stats: > COMPLETE Column stats: NONE > Filter Operator > predicate: id is not null (type: boolean) > Statistics: Num rows: 1 Data size: 66 Basic stats: > COMPLETE Column stats: NONE > Map Join Operator > condition map: > Inner Join 0 to 1 > keys: > 0 id (type: int) > 1 _col0 (type: int) > outputColumnNames: _col0, _col1, _col2, _col3, > _col7, _col8 > input vertices: > 1 Map 1 > Statistics: Num rows: 6 Data size: 52 Basic stats: > COMPLETE Column stats: NONE > Select Operator > expressions: _col0 (type: int), _col1 (type: > string), _col2 (type: int), _col3 (type: int), _col7 (type: int), _col8 > (type: string) > outputColumnNames: _col0, _col1, _col2, _col3, > _col4, _col5 > Statistics: Num rows: 6 Data size: 52 Basic stats: > COMPLETE Column stats: NONE > File Output Operator > compressed: false > Statistics: Num rows: 6 Data size: 52 Basic > stats: COMPLETE Column stats: NONE > table: > input format: > org.apache.hadoop.mapred.TextInputFormat > output format: > org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat > serde: > org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe > Local Work: > Map Reduce Local Work > > Stage: Stage-0 > Fetch Operator > limit: -1 > Processor Tree: > ListSink > > > > *I can't find any information about union "test" in explain 2.* > > *Some properties on hive-site.xml is * > > hive.ignore.mapjoin.hint > false > > > hive.auto.convert.join > true > > > hive.auto.convert.join.noconditionaltask > true > > > > > *Thanks.* > *Best wishes.* > --047d7b5d49de7935b105277b10f2 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hive 1.2.1 on Spark1.4.1
<= br>
2015-12-22 19:31 GMT+08:00 Jone Zhang <= joyoungzhang@gmail.com>:
select =C2=A0* from staff;
1 jone 22 1
2 lucy 21 1
3 hmm 22 2
<= div>4 james 24 3
5 xiaoliu 23 <= /span>3

select id,date_ from trade union all se= lect id,"test" from trade ;
1 201510210908
2 201509080234
2 201509080235
1 test
2 test
2 test

set hive.execution.engine=3Dspark;
set spar= k.master=3Dlocal;
select /*+mapjoin(t)*/ * from staf= f s join=C2=A0
(select id,date_ from trade union all selec= t id,"test" from trade ) t on s.id=3Dt.id;
1 jone 22 1 1 201510210908
2 lucy 21 1 2 201509080234
2= lucy 21 1 2= 201509080235

set hive.execution.en= gine=3Dmr;
select /*+mapjoin(t)*/ * from staff s join=C2= =A0
(select id,date_ from trade union all select id,"= test" from trade ) t on s.id= =3Dt.id;
<= div>FAILED: SemanticException [Error 10227]: Not all clauses are supported = with mapjoin hint. Please remove mapjoin hint.

I have two questions
<= div>1.Why result of hive on spar= k not include the following record?
1 jone 22 1 1 test
2= lucy 21 1 2= test
2 lucy 21 1 2 test

2.Why there are two different ways of dealing same query?


explain 1:
set hive.exec= ution.engine=3Dspark;
set spark.master=3Dlocal;
<= div>explain=C2=A0
select id,date_ from trade union all = select id,"test" from trade;
OK
STAGE DEP= ENDENCIES:
=C2=A0 Stage-1 is a root stage
=C2=A0 Stage-= 0 depends on stages: Stage-1

STAGE PLANS:
=C2=A0 Stage: Stage-1
=C2=A0 =C2=A0 Spark
=C2=A0 =C2= =A0 =C2=A0 DagName: jonezhang_20151222191643_5301d90a-caf0-4934-8092-d165c8= 7a4190:1
=C2=A0 =C2=A0 =C2=A0 Vertices:
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 Map 1=C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 Map Operator Tree:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 TableScan
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 alias: trade
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num rows: 6 Data size: 48 Ba= sic stats: COMPLETE Column stats: NONE
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Select Operator
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 expressions: id= (type: int), date_ (type: string)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 outputColumnNames: _col0, _col1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 Statistics: Num rows: 6 Data size: 48 Basic stats: COMPLETE Column stat= s: NONE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 File Output Operator
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 compressed: false
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 Statistics: Num rows: 12 Data size: 96 Basic stats: COMPLETE Column sta= ts: NONE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 table:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 input format: org.a= pache.hadoop.mapred.TextInputFormat
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 output forma= t: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 serde: org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Map 2=C2=A0
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 Map Operator Tree:
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TableScan
=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 alias: trade
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num row= s: 6 Data size: 48 Basic stats: COMPLETE Column stats: NONE
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Select Operator=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 expressions: id (type: int), 'test' (type: string)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 outp= utColumnNames: _col0, _col1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num rows: 6 Data size: 48 Ba= sic stats: COMPLETE Column stats: NONE
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 File Output Operator
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 compressed: false
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num rows: 12 Data size: 96 B= asic stats: COMPLETE Column stats: NONE
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 table:
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 input format: org.apache.hadoop.mapred.TextInputFormat
<= div>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 output format: org.apache.hadoop.hive.ql.io.HiveIgnore= KeyTextOutputFormat
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 serde: org.apache.hadoop.hive= .serde2.lazy.LazySimpleSerDe

=C2=A0 Stage: Stage-0=
=C2=A0 =C2=A0 Fetch Operator
=C2=A0 =C2=A0 =C2=A0 limi= t: -1
=C2=A0 =C2=A0 =C2=A0 Processor Tree:
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 ListSink


expla= in 2:
set hive.execution.engine=3Dspark;
= set spark.master=3Dlocal;
explain=C2=A0
s= elect /*+mapjoin(t)*/ * from staff s join=C2=A0
(select id= ,date_ from trade union all select id,"20999999999" from trade ) = t on s.id=3Dt.id;
OK
STAGE DEPENDE= NCIES:
=C2=A0 Stage-2 is a root stage
=C2=A0 Stage-1 de= pends on stages: Stage-2
=C2=A0 Stage-0 depends on stages: Stage-= 1

STAGE PLANS:
=C2=A0 Stage: Stage-2
=C2=A0 =C2=A0 Spark
=C2=A0 =C2=A0 =C2=A0 DagName: jonezhan= g_20151222191716_be7eac84-b5b6-4478-b88f-9f59e2b1b1a8:3
=C2=A0 = =C2=A0 =C2=A0 Vertices:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Map 1=C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Map Operator Tree:
=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TableScan
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 alias:= trade
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 Statistics: Num rows: 6 Data size: 48 Basic stats: COMPLETE Column s= tats: NONE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Filter Operator
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 predicate: id is not null (type: boolean)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 Statistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stat= s: NONE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 Select Operator
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 expressions: id (type: int), date= _ (type: string)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 outputColumnNames: _col0, _col1
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 S= tatistics: Num rows: 3 Data size: 24 Basic stats: COMPLETE Column stats: NO= NE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 Spark HashTable Sink Operator
=C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 keys:
=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 0 id (type: int)
=C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 1 _col0 (ty= pe: int)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Local Work:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Map Reduce Local W= ork

=C2=A0 Stage: Stage-1
=C2=A0 =C2=A0 = Spark
=C2=A0 =C2=A0 =C2=A0 DagName: jonezhang_20151222191716_be7e= ac84-b5b6-4478-b88f-9f59e2b1b1a8:2
=C2=A0 =C2=A0 =C2=A0 Vertices:=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 Map 2=C2=A0
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 Map Operator Tree:
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 TableScan
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 alias: s
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num row= s: 1 Data size: 66 Basic stats: COMPLETE Column stats: NONE
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Filter Operator=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 predicate: id is not null (type: boolean)
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num row= s: 1 Data size: 66 Basic stats: COMPLETE Column stats: NONE
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Map Join= Operator
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 condition map:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Inner = Join 0 to 1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 keys:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 0 id (type: int)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 1 _col0 (type: int)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 outputColumnNames: _col0, _col1,= _col2, _col3, _col7, _col8
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 input vertices:
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 1 Map 1
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num rows: 6 Data size: 52 Basic stats:= COMPLETE Column stats: NONE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Select Operator
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 expressions: _col0 (type: int), _col1 (type: string), _col2 (type: int)= , _col3 (type: int), _col7 (type: int), _col8 (type: string)
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 outputColumnNames: _col0, _col1, _col2, _col3, _col4, _col5
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Statistics: Num rows: 6 Data size: 52 Basic stats: COMPLETE Colu= mn stats: NONE
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 File Output Operator
=C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 compressed: false
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Statistics: Num row= s: 6 Data size: 52 Basic stats: COMPLETE Column stats: NONE
=C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 table:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 input format= : org.apache.hadoop.mapred.TextInputFormat
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 output format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOut= putFormat
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 serde: org.apache.hadoop.= hive.serde2.lazy.LazySimpleSerDe
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 Local Work:
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 Map Reduce Local Work

=C2=A0 Stage: Stage-0=
=C2=A0 =C2=A0 Fetch Operator
=C2=A0 =C2=A0 =C2=A0 limi= t: -1
=C2=A0 =C2=A0 =C2=A0 Processor Tree:
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 ListSink



I can't find any information about union "test" in = explain 2.

Some properties on hive-s= ite.xml is=C2=A0
<property>
<name>hive.= ignore.mapjoin.hint</name>
<value>false</value>=
</property>
<property>
<name&= gt;hive.auto.convert.join</name>
<value>true</valu= e>
</property>
<property>
<= name>hive.auto.convert.join.noconditionaltask</name>
<= ;value>true</value>




Thanks.
Best wishes.=

--047d7b5d49de7935b105277b10f2--