hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xiaoyong Zhu <xiaoy...@microsoft.com>
Subject RE: relationship between a hive query and a tez dag
Date Fri, 19 Dec 2014 06:39:38 GMT
Sorry to update this again - but why don't we do a cross query optimization and make the query
into 1 DAG (if all the queries in a certain script are linked with each other).. this seems
a more optimized way..

Xiaoyong

From: Xiaoyong Zhu [mailto:xiaoyzhu@microsoft.com]
Sent: Thursday, December 18, 2014 8:08 AM
To: Alan Gates
Cc: user@hive.apache.org
Subject: RE: relationship between a hive query and a tez dag

I see, thanks for the explaination!

Xiaoyong

From: Alan Gates [mailto:gates@hortonworks.com]
Sent: Thursday, December 18, 2014 8:05 AM
To: Xiaoyong Zhu
Cc: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: relationship between a hive query and a tez dag

DDL operations are done from the client, not as part of a Tez DAG.  There still is no linkage
between the DAGs even if they access the same tables or somehow depend on each other.  Hive
manages making sure queries are executed in the order given.  Tez just executed each query.

Alan.

[cid:image001.jpg@01D01B99.9B4DD560]
Xiaoyong Zhu<mailto:xiaoyzhu@microsoft.com>
December 16, 2014 at 15:41
Thanks Alan! Then what if different hive queries have some links (e.g. query 1 creates table
abc and query 2 reads data from abc) will this be in 1 DAG or 2 DAGs? If 2, then is there
any link/relatioinship between the 2 DAGs?

Xiaoyong

From: Alan Gates [mailto:gates@hortonworks.com]
Sent: Wednesday, December 17, 2014 5:27 AM
To: user@hive.apache.org<mailto:user@hive.apache.org>
Subject: Re: relationship between a hive query and a tez dag

Several.  Each Hive query becomes one Tez DAG.

Alan.



--
Sent with Postbox<http://www.getpostbox.com>

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed
and may contain information that is confidential, privileged and exempt from disclosure under
applicable law. If the reader of this message is not the intended recipient, you are hereby
notified that any printing, copying, dissemination, distribution, disclosure or forwarding
of this communication is strictly prohibited. If you have received this communication in error,
please contact the sender immediately and delete it from your system. Thank You.
[cid:image002.jpg@01D01B99.9B4DD560]
Alan Gates<mailto:gates@hortonworks.com>
December 16, 2014 at 13:26
Several.  Each Hive query becomes one Tez DAG.

Alan.

--
Sent with Postbox<http://www.getpostbox.com>

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to which it is addressed
and may contain information that is confidential, privileged and exempt from disclosure under
applicable law. If the reader of this message is not the intended recipient, you are hereby
notified that any printing, copying, dissemination, distribution, disclosure or forwarding
of this communication is strictly prohibited. If you have received this communication in error,
please contact the sender immediately and delete it from your system. Thank You.

Mime
View raw message