Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 331CC11B92 for ; Tue, 26 Aug 2014 01:19:42 +0000 (UTC) Received: (qmail 58010 invoked by uid 500); 26 Aug 2014 01:19:40 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 57942 invoked by uid 500); 26 Aug 2014 01:19:40 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 57932 invoked by uid 99); 26 Aug 2014 01:19:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Aug 2014 01:19:40 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rgrandl@yahoo.com designates 216.109.114.219 as permitted sender) Received: from [216.109.114.219] (HELO nm47-vm10.bullet.mail.bf1.yahoo.com) (216.109.114.219) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Aug 2014 01:19:35 +0000 Received: from [66.196.81.174] by nm47.bullet.mail.bf1.yahoo.com with NNFMP; 26 Aug 2014 01:19:13 -0000 Received: from [98.139.212.216] by tm20.bullet.mail.bf1.yahoo.com with NNFMP; 26 Aug 2014 01:19:13 -0000 Received: from [127.0.0.1] by omp1025.mail.bf1.yahoo.com with NNFMP; 26 Aug 2014 01:19:13 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 677775.15742.bm@omp1025.mail.bf1.yahoo.com Received: (qmail 24623 invoked by uid 60001); 26 Aug 2014 01:19:13 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1409015953; bh=mWTkWuPU6UuyEl8nvGy+OJva48xcafjq0fTA1LMsRQg=; h=References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=59bnKNEUQofiszk3NtJ+GmwnbcQ9bPiLy+BqpzVLwR8OJKijOS0zjA+CdUT0dzFdVXBAaL6OTkfipFOvQomTiueT5kUITENrwKJwkEN0cDgQSaCPFBZ0Pip70Z+xuxkdGfCYLakuDrL0uh8cqFq0+mWKF/grAgR0ZDrHbgeJK7o= X-YMail-OSG: 7vlX8noVM1mJAMAi1CO2FmBq0Qwbctz7dgmz0I6DKl0Y4hv NMzWSkfJE8fNsJWnq7QvPkFpb1Lq1onbUut86cdO5S5xNC9prrcJ_1Sk92PE yDIlQ0w_kIs8.UQIq9aPFqbT9Ic5jESfCo1N.7bwHgwp6rEQVqtlDQ9Puhlq 6VOJbhDyfRm.xRESFwJst0KGylLPeEcpenbc0uw6GSPND2JKvanMwUuWURlw e.dsALRld6a4Z1m8liq2Pv02ywrmLK94nHP_BVFv6Mhe56Yi_46odL_QKA_s E8kmp8GQ9nrp1Y7xABIAio1Nn7SQk0OUnsxF.7UNtLQmWUPw6YnjFFlAtYIL SpTnyLL8egiS5rjl4T3J3Q9RXp19FT9UXJY8UacY9lSJs129yREyffU_SEWJ TGUrGc8_YLP6mK1eZXsDfjnUZ1opTyWm19Dr53EuggYEGfSdWVXclVuZ5Kat O.HXtGqTkNd7NlYnsV7uT7RXwXjKnheGzMX6yj384l873vETcCBdkWsqh9qM 16irFeej.Y_T7tH43U3iZli42_mv3r2H8mLjSOEOgpbruBPLbB9uMinTg5B6 PZlTOhfMMRyWOGzKfgKxtXgEdFxZloEcWhqKE.Pdx99C1ZE1qiol2e4YGj9. lVHCIguFgUQJgsjYy4UkIUp.uLRYE3X_ajr5QlNkXY8A- Received: from [131.107.147.221] by web161603.mail.bf1.yahoo.com via HTTP; Mon, 25 Aug 2014 18:19:13 PDT X-Rocket-MIMEInfo: 002.001,SSBjcmVhdGVkIGFub3RoZXIgdmFyaWFibGUgaGl2ZS5qb2IubmFtZSBhbmQgc2V0IGl0IGZyb20gc3FsIHNjcmlwdCBhbmQgd29ya3MuIAoKClRoYW5rcywKUm9iZXJ0CgoKCk9uIE1vbmRheSwgQXVndXN0IDI1LCAyMDE0IDU6NDAgUE0sIEdyYW5kbCBSb2JlcnQgPHJncmFuZGxAeWFob28uY29tPiB3cm90ZToKIAoKCkhpIGd1eXMsCgpJIGFtIHN0aWxsIHN0cnVnZ2xpbmcgd2l0aCBmaW5kaW5nIGEgd2F5IHRvIHNwZWNpZnkgYSBqb2IgbmFtZSBpbiB0aGUgaGl2ZSBxdWVyeSBhbmQgcHJvcGFnYXRlIHRoYXQBMAEBAQE- X-Mailer: YahooMailWebService/0.8.201.700 References: <1404963838.14559.YahooMailNeo@web161603.mail.bf1.yahoo.com> <1409013607.83107.YahooMailNeo@web161601.mail.bf1.yahoo.com> Message-ID: <1409015953.23241.YahooMailNeo@web161603.mail.bf1.yahoo.com> Date: Mon, 25 Aug 2014 18:19:13 -0700 From: Grandl Robert Reply-To: Grandl Robert Subject: Re: pass new job name to tez To: "user@hive.apache.org" , =?utf-8?B?TmF2aXPrpZjsirnsmrA=?= , Grandl Robert In-Reply-To: <1409013607.83107.YahooMailNeo@web161601.mail.bf1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-2012143828-1050960132-1409015953=:23241" X-Virus-Checked: Checked by ClamAV on apache.org ---2012143828-1050960132-1409015953=:23241 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I created another variable hive.job.name and set it from sql script and wor= ks. =0A=0A=0AThanks,=0ARobert=0A=0A=0A=0AOn Monday, August 25, 2014 5:40 PM= , Grandl Robert wrote:=0A =0A=0A=0AHi guys,=0A=0AI am s= till struggling with finding a way to specify a job name in the hive query = and propagate that name in Tez dag. For example my hive script is named que= ry55.=0A=0ADo you know a way I can pass this name to be available in Tez da= g ?=0A=0ANavis suggested to look at hive.query.id. I tried in my query55.sq= l to do a set hive.query.id=3Dquery55; but the name did not change actually= .=0A=0ADid someone try this before, or have more suggestions about an easy = way to do it ? =0A=0A=0AActually what I want to achieve here is to correlat= e a tez dag with a query name. So have a tez dag name to be specified by th= e user, and hence comes through the hive query.=0A=0AThanks,=0ARobert=0A=0A= =0AOn Wednesday, July 9, 2014 8:56 PM, Navis=EB=A5=98=EC=8A=B9=EC=9A=B0 wrote:=0A =0A=0A=0AIn=C2=A0GenTezProcContext, you can fin= d "new TezWork(conf.getVar(HiveConf.ConfVars.HIVEQUERYID))" part. And the f= inal name is name + ":" + (++counter);=0A=0AThanks,=0ANavis=0A=0A=0A=0A2014= -07-10 12:43 GMT+09:00 Grandl Robert :=0A=0AHi guys,=0A>= =0A>=0A>I am trying to identify a DAG in Tez with a different id, based on = job =0Aname(for e.g. query55.sql from hive-testbench) + input size. =0A>=0A= >=0A>=0A>So my new identifier should be for example query55_2048MB. It seem= s that a DAG in tez, already takes a name which comes from a jobPlan.getNam= e() =0Apassed through google protobuf thing. Because the DAG is generated i= n Hive, I think the new identifier should come from Hive right ?=0A>=0A>=0A= >Can you pinpoint me which classes, I should change in Hive in order to pro= pagate the new identifier for the DAG ?=0A>=0A>=0A>Thanks,=0A>Robert=0A>=0A= > ---2012143828-1050960132-1409015953=:23241 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
I created = another variable hive.job.name and set it from sql script and works.

Thanks,<= /div>
Robert
<= br>
On Monday, August 25, 2014 5:40 PM, Grandl Robert <rg= randl@yahoo.com> wrote:


Hi guys,

I am still struggling with finding a way to specify a job= name in the hive query and propagate that name in Tez dag. For example my = hive script is named query55.


Navis sugg= ested to look at hive.query.id. I tried in my query55.sql to do a set hive.= query.id=3Dquery55; but the name did not change actually.
=
Did someone try this before= , or have more suggestions about an easy way to do it ?
=

Actually w= hat I want to achieve here is to correlate a tez dag with a query name. So = have a tez dag name to be specified by the user, and hence comes through th= e hive query.

Tha= nks,
Robert


On Wednesday, July 9, 2014 8:56 PM, Navis=EB=A5=98=EC=8A=B9=EC=9A= =B0 <navis.ryu@nexr.com> wrote:

In GenTezProcCon= text, you can find "new TezWork(conf.getVar(HiveConf.ConfVars.HIVEQUERYID))= " part. And the final name is name + ":" + (++counter);

Thanks,
Navis
=0A


2014-07-10 12:43 GMT+09:00 Grandl Robert <rgrandl@yahoo.com= >:
=0A
=0A
Hi guys,

I=0A am trying to identify a DAG in Tez with a different = id, based on job =0Aname(for e.g. query55.sql from hive-testbench) + input = size.

=0A
So=0A my new= identifier should be for example query55_2048MB. It seems that a=0A DAG in= tez, already takes a name which comes from a jobPlan.getName() =0Apassed t= hrough google protobuf=0A thing. Because the DAG is generated in Hive, I th= ink the new identifier=0A should come from Hive right ?

=0ACan you pinpoint me which classes, I should change in Hiv= e in order to propagate the new identifier for the DAG ?
=0A
Thanks,
=0ARobert
=

<= br clear=3D"none">


<= /div>


=
---2012143828-1050960132-1409015953=:23241--