Return-Path: X-Original-To: apmail-hive-user-archive@www.apache.org Delivered-To: apmail-hive-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3F0D711A53 for ; Tue, 26 Aug 2014 00:40:37 +0000 (UTC) Received: (qmail 78641 invoked by uid 500); 26 Aug 2014 00:40:35 -0000 Delivered-To: apmail-hive-user-archive@hive.apache.org Received: (qmail 78565 invoked by uid 500); 26 Aug 2014 00:40:35 -0000 Mailing-List: contact user-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hive.apache.org Delivered-To: mailing list user@hive.apache.org Received: (qmail 78555 invoked by uid 99); 26 Aug 2014 00:40:35 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Aug 2014 00:40:35 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rgrandl@yahoo.com designates 98.139.213.148 as permitted sender) Received: from [98.139.213.148] (HELO nm27-vm1.bullet.mail.bf1.yahoo.com) (98.139.213.148) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Aug 2014 00:40:28 +0000 Received: from [98.139.212.152] by nm27.bullet.mail.bf1.yahoo.com with NNFMP; 26 Aug 2014 00:40:07 -0000 Received: from [98.139.212.210] by tm9.bullet.mail.bf1.yahoo.com with NNFMP; 26 Aug 2014 00:40:07 -0000 Received: from [127.0.0.1] by omp1019.mail.bf1.yahoo.com with NNFMP; 26 Aug 2014 00:40:07 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 763972.47516.bm@omp1019.mail.bf1.yahoo.com Received: (qmail 98995 invoked by uid 60001); 26 Aug 2014 00:40:07 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1409013607; bh=o1wZN+ZznmE/fDsx9zearXr8c9eAJfRWoPqGABMzTPM=; h=References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type; b=GNgHCTNQ94fOhZI4WbD6fCuBnwDsNMCmbNuxMn0h0W6kD0DcAXieXmtdsQJ1aJ28s3CLTbTNWH8o3mMWUNbRmN/Nx4niWvFmlIyq7L7psUocnBfXbqEEPnA0pjkXpfEPKc2aHFLkmZtIHVL/v8zNh67wX9cUnZsZh8UnJ/cjIII= X-YMail-OSG: QPfn_nIVM1mvgPi1bbDSamEr1NjmJNtilNr7XRj4c2s01cx limENf_Z2u.R.sJc2hS5BL72OSyolw3pUq0NS9tJ5fomd5j929RbDImLTdf9 hBJ4s9hpoIsiVHzyyXiBB2Y5CLH2ut3oMHCjwWyONnV0G4KqE1Vyrw4j0aOo ejLqJn0eNhuAQSwFZA9.4WbPxs91TI.Mx_mmcTX0ywLQgvT_X1DZYQ6OyUqC x8sW8Q7jXyZUXNkBG24QhanvRJLv2XbNsDlixcpvndtqR7pWIIcypPUifNVo 7atMDqy2ecrr3WOL0JJAz5oLHYXQWetwsx.Je7mRzsnEega0JkST36gPTmuN qziP.7ra6v1nKYJnco2XqnQZ6.900O3LsRgbW0wx4DcU6.Y5FYmfNTxMx6je LCmeqSIQYT74dlpPGBcPCDDzuztsd5GCDE5fhbRUIX5HAOLZJkrg1RNOBCzm w0ZmsusRir8v3qqTMDD1ldJMN3aH7IsFAtICoznGOBiuC2mZd1c7.6cA6i11 2YkWnwOVA4Tp7QE8ueNooqgq7PcYkhA9c910fcdSzEDHe3coAvBQUnujXGwJ w0POrqvIuACStXiuBObLJBTdKoKyEfb8DEUR707UHkGFOipd_OLUSHGyh.xd Gg.IMRawWoYv6MGLN8mf_MOlqB8x43tH_o.6DJwJ30sYv Received: from [131.107.174.221] by web161601.mail.bf1.yahoo.com via HTTP; Mon, 25 Aug 2014 17:40:07 PDT X-Rocket-MIMEInfo: 002.001,SGkgZ3V5cywKCkkgYW0gc3RpbGwgc3RydWdnbGluZyB3aXRoIGZpbmRpbmcgYSB3YXkgdG8gc3BlY2lmeSBhIGpvYiBuYW1lIGluIHRoZSBoaXZlIHF1ZXJ5IGFuZCBwcm9wYWdhdGUgdGhhdCBuYW1lIGluIFRleiBkYWcuIEZvciBleGFtcGxlIG15IGhpdmUgc2NyaXB0IGlzIG5hbWVkIHF1ZXJ5NTUuCgpEbyB5b3Uga25vdyBhIHdheSBJIGNhbiBwYXNzIHRoaXMgbmFtZSB0byBiZSBhdmFpbGFibGUgaW4gVGV6IGRhZyA_CgpOYXZpcyBzdWdnZXN0ZWQgdG8gbG9vayBhdCBoaXZlLnF1ZXJ5LmlkLiBJIHRyaWUBMAEBAQE- X-Mailer: YahooMailWebService/0.8.201.700 References: <1404963838.14559.YahooMailNeo@web161603.mail.bf1.yahoo.com> Message-ID: <1409013607.83107.YahooMailNeo@web161601.mail.bf1.yahoo.com> Date: Mon, 25 Aug 2014 17:40:07 -0700 From: Grandl Robert Reply-To: Grandl Robert Subject: Re: pass new job name to tez To: =?utf-8?B?TmF2aXPrpZjsirnsmrA=?= , "user@hive.apache.org" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-83809723-434650337-1409013607=:83107" X-Virus-Checked: Checked by ClamAV on apache.org ---83809723-434650337-1409013607=:83107 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi guys,=0A=0AI am still struggling with finding a way to specify a job nam= e in the hive query and propagate that name in Tez dag. For example my hive= script is named query55.=0A=0ADo you know a way I can pass this name to be= available in Tez dag ?=0A=0ANavis suggested to look at hive.query.id. I tr= ied in my query55.sql to do a set hive.query.id=3Dquery55; but the name did= not change actually.=0A=0ADid someone try this before, or have more sugges= tions about an easy way to do it ? =0A=0A=0AActually what I want to achieve= here is to correlate a tez dag with a query name. So have a tez dag name t= o be specified by the user, and hence comes through the hive query.=0A=0ATh= anks,=0ARobert=0A=0A=0A=0AOn Wednesday, July 9, 2014 8:56 PM, Navis=EB=A5= =98=EC=8A=B9=EC=9A=B0 wrote:=0A =0A=0A=0AIn=C2=A0GenTe= zProcContext, you can find "new TezWork(conf.getVar(HiveConf.ConfVars.HIVEQ= UERYID))" part. And the final name is name + ":" + (++counter);=0A=0AThanks= ,=0ANavis=0A=0A=0A=0A2014-07-10 12:43 GMT+09:00 Grandl Robert :=0A=0AHi guys,=0A>=0A>=0A>I am trying to identify a DAG in Tez with = a different id, based on job =0Aname(for e.g. query55.sql from hive-testben= ch) + input size. =0A>=0A>=0A>=0A>So my new identifier should be for exampl= e query55_2048MB. It seems that a DAG in tez, already takes a name which co= mes from a jobPlan.getName() =0Apassed through google protobuf thing. Becau= se the DAG is generated in Hive, I think the new identifier should come fro= m Hive right ?=0A>=0A>=0A>Can you pinpoint me which classes, I should chang= e in Hive in order to propagate the new identifier for the DAG ?=0A>=0A>=0A= >Thanks,=0A>Robert=0A>=0A> ---83809723-434650337-1409013607=:83107 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi guys,

I am still stru= ggling with finding a way to specify a job name in the hive query and propa= gate that name in Tez dag. For example my hive script is named query55.

Do you know a wa= y I can pass this name to be available in Tez dag ?
=
Navis suggested to look at hive.quer= y.id. I tried in my query55.sql to do a set hive.query.id=3Dquery55; but th= e name did not change actually.

Did someone try this before, or have more suggestions about= an easy way to do it ?

<= div style=3D"color: rgb(0, 0, 0); font-size: 13.3333px; font-family: times = new roman,new york,times,serif; background-color: transparent; font-style: = normal;">Actually what I want to achieve here is to correlate a tez d= ag with a query name. So have a tez dag name to be specified by the user, a= nd hence comes through the hive query.

Thanks,
Robert


On Wedne= sday, July 9, 2014 8:56 PM, Navis=EB=A5=98=EC=8A=B9=EC=9A=B0 <navis.ryu@= nexr.com> wrote:


In GenTezProcCont= ext, you can find "new TezWork(conf.getVar(HiveConf.ConfVars.HIVEQUERYID))"= part. And the final name is name + ":" + (++counter);

Thanks,
Navis
=0A


2014-07-10 12:43 GMT+09:00 Grandl Robert <= span dir=3D"ltr"><rgran= dl@yahoo.com>:
=0A
<= div>
=0A
Hi guys,

I=0A am trying to identify a DAG in Tez with a= different id, based on job =0Aname(for e.g. query55.sql from hive-testbenc= h) + input size.

=0A
S= o=0A my new identifier should be for example query55_2048MB. It seems that = a=0A DAG in tez, already takes a name which comes from a jobPlan.getName() = =0Apassed through google protobuf=0A thing. Because the DAG is generated in= Hive, I think the new identifier=0A should come from Hive right ?
<= br clear=3D"none">
=0ACan you pinpoint me which classes, I should ch= ange in Hive in order to propagate the new identifier for the DAG ?
= =0A
Thanks,
=0ARobert




---83809723-434650337-1409013607=:83107--