hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gopal Vijayaraghavan <gop...@apache.org>
Subject Re: unique-id for the mapper task with tez execution engine
Date Wed, 23 Dec 2015 05:28:50 GMT
Hi,

(x-posts to bcc:)

On 12/22/15, 9:19 PM, "Amey Barve" <ameybarve15@gmail.com> wrote:

>conf.get("mapreduce.task.id");
>
>Now I want to run same hive queries with tez engine and I want to know
>what
>should be my unique-id. Is there any property from configuration or other
>that can give me unique-id from the mapper task?
>
>I am using conf.get("mapreduce.task.partition") property, Is that
>guranteed
>to give unique - id for tez?

That's populated in Tez in pretty much the exact format used by MRv2.

https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hi
ve/ql/exec/tez/TezProcessor.java#L112


But beware, each mapper & reducer think it's the only one of its kind out
there, so there are collisions between DAG stages for the unique name.

Cheers,
Gopal



Mime
View raw message