Mailing-List: contact user-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hive.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAOGjC_5-7m=euFuk83tcrxNPngJj1NqQ2G9ZME=ZRJsFqEx9Nw@mail.gmail.com>
References: 
 <CAOGjC_5-7m=euFuk83tcrxNPngJj1NqQ2G9ZME=ZRJsFqEx9Nw@mail.gmail.com>
Date: Fri, 6 Jul 2012 09:19:21 +0900
Message-ID: 
 <CACQ46vHpn3WxGX5-+x7BF37r=d0dbqHHYD7WMuGRnipOHrLX4Q@mail.gmail.com>
Subject: Re: Pulling off server's debug info to Client
From: =?EUC-KR?B?TmF2aXO3+b3Cv+w=?= <navis.ryu@nexr.com>
To: user@hive.apache.org
Content-Type: multipart/alternative; boundary=047d7b2ee18b3b8feb04c41e3720

--047d7b2ee18b3b8feb04c41e3720
Content-Type: text/plain; charset=ISO-8859-1

I've modified hive to notify jobId for monitoring purpose. Check the issue
and vote for acceptance ^^

https://issues.apache.org/jira/browse/HIVE-3235

2012/7/6 VanHuy Pham <huy.pham.sc@gmail.com>

> Hi all,
>     I have been interested in getting the debug information and display it
> to clients.
>     Specifically, when you submit a job to hive cli (on terminal say you
> write "select * from somewhere ...) or through a thrift server, you will
> see they hive displays some debug information on the screen (See below)
>
>     How I can I get this "live" information to the client while the
> job is running so the client can see the stages of the job and how it
> is going, see if the job gets killed, or fails... I am using thrift
> server, and see this information on the thrift server's terminal and
> wonder if I can transfer this info to thrift client. Or if it's
> impossible to transfer this info to thrift client, how would one
> direct this debug info somewhere to local file or mysql then the
> information can be read from there?
>
> Any ideas?
> Thanks
>
>
>
> ------------------------------------------------------
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> ---> getPaths = XXXXXXXXXXXX
> Starting Job = XXXXXXXXXXXXXX, Tracking URL = XXXXXXXXXXXXXXXXXX
> Kill Command = XXXXXXXXXXXXXXXXXXXX
> Hadoop job information for Stage-1: number of mappers: 505; number of
> reducers: 0
> 2012-07-04 01:52:42,145 Stage-1 map = 0%,  reduce = 0%
> 2012-07-04 01:52:48,185 Stage-1 map = 5%,  reduce = 0%, Cumulative CPU
> 42.92 sec
> 2012-07-04 01:52:49,196 Stage-1 map = 26%,  reduce = 0%, Cumulative
> CPU 213.03 sec
>

--047d7b2ee18b3b8feb04c41e3720
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

I&#39;ve modified hive to notify jobId for monitoring purpose. Check the is=
sue and vote for acceptance ^^<div><br></div><div><a href=3D"https://issues=
.apache.org/jira/browse/HIVE-3235">https://issues.apache.org/jira/browse/HI=
VE-3235</a><br>
<br><div class=3D"gmail_quote">2012/7/6 VanHuy Pham <span dir=3D"ltr">&lt;<=
a href=3D"mailto:huy.pham.sc@gmail.com" target=3D"_blank">huy.pham.sc@gmail=
.com</a>&gt;</span><br><blockquote class=3D"gmail_quote" style=3D"margin:0 =
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi all,<br>
=A0 =A0 I have been interested in getting the debug information and display=
 it<br>
to clients.<br>
=A0 =A0 Specifically, when you submit a job to hive cli (on terminal say yo=
u<br>
write &quot;select * from somewhere ...) or through a thrift server, you wi=
ll<br>
see they hive displays some debug information on the screen (See below)<br>
<br>
=A0 =A0 How I can I get this &quot;live&quot; information to the client whi=
le the<br>
job is running so the client can see the stages of the job and how it<br>
is going, see if the job gets killed, or fails... I am using thrift<br>
server, and see this information on the thrift server&#39;s terminal and<br=
>
wonder if I can transfer this info to thrift client. Or if it&#39;s<br>
impossible to transfer this info to thrift client, how would one<br>
direct this debug info somewhere to local file or mysql then the<br>
information can be read from there?<br>
<br>
Any ideas?<br>
Thanks<br>
<br>
<br>
<br>
------------------------------------------------------<br>
Total MapReduce jobs =3D 1<br>
Launching Job 1 out of 1<br>
Number of reduce tasks is set to 0 since there&#39;s no reduce operator<br>
---&gt; getPaths =3D XXXXXXXXXXXX<br>
Starting Job =3D XXXXXXXXXXXXXX, Tracking URL =3D XXXXXXXXXXXXXXXXXX<br>
Kill Command =3D XXXXXXXXXXXXXXXXXXXX<br>
Hadoop job information for Stage-1: number of mappers: 505; number of<br>
reducers: 0<br>
2012-07-04 01:52:42,145 Stage-1 map =3D 0%, =A0reduce =3D 0%<br>
2012-07-04 01:52:48,185 Stage-1 map =3D 5%, =A0reduce =3D 0%, Cumulative CP=
U 42.92 sec<br>
2012-07-04 01:52:49,196 Stage-1 map =3D 26%, =A0reduce =3D 0%, Cumulative<b=
r>
CPU 213.03 sec<br>
</blockquote></div><br></div>

--047d7b2ee18b3b8feb04c41e3720--