Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B0A1B101FC for ; Thu, 2 Jan 2014 16:34:35 +0000 (UTC) Received: (qmail 17687 invoked by uid 500); 2 Jan 2014 16:32:41 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 17610 invoked by uid 500); 2 Jan 2014 16:32:37 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 17563 invoked by uid 99); 2 Jan 2014 16:32:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Jan 2014 16:32:20 +0000 X-ASF-Spam-Status: No, hits=-2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_HI,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [211.189.100.14] (HELO usmailout4.samsung.com) (211.189.100.14) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Jan 2014 16:32:12 +0000 Received: from uscpsbgm2.samsung.com (u115.gpu85.samsung.co.kr [203.254.195.115]) by usmailout4.samsung.com (Oracle Communications Messaging Server 7u4-24.01(7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTP id <0MYS00LU58L2G780@usmailout4.samsung.com> for user@hadoop.apache.org; Thu, 02 Jan 2014 11:31:50 -0500 (EST) X-AuditID: cbfec373-b7f4a6d000005e0a-f3-52c594760226 Received: from ussync1.samsung.com ( [203.254.195.81]) by uscpsbgm2.samsung.com (USCPMTA) with SMTP id FE.5D.24074.67495C25; Thu, 02 Jan 2014 11:31:50 -0500 (EST) Received: from lgflarrahondo ([105.140.33.168]) by ussync1.samsung.com (Oracle Communications Messaging Server 7u4-23.01 (7.0.4.23.0) 64bit (built Aug 10 2011)) with ESMTPA id <0MYS00JE78L18V90@ussync1.samsung.com> for user@hadoop.apache.org; Thu, 02 Jan 2014 11:31:50 -0500 (EST) From: German Florez-Larrahondo To: user@hadoop.apache.org References: <52C1C75B.4060800@gmail.com> In-reply-to: Subject: RE: get job data in command line in MRv2 Date: Thu, 02 Jan 2014 10:31:49 -0600 Message-id: <016601cf07d8$23328ec0$6997ac40$@samsung.com> MIME-version: 1.0 Content-type: multipart/alternative; boundary="----=_NextPart_000_0167_01CF07A5.D89A1A90" X-Mailer: Microsoft Outlook 14.0 Thread-index: AQGPRRruDNYN/16kmPwRGFC6OpYiLwID47tlAvPGhQKayWgEwA== Content-language: en-us X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrHLMWRmVeSWpSXmKPExsVy+t/hQN2yKUeDDD5tZ7PomTKNxYHRY0LX FsYAxigum5TUnMyy1CJ9uwSujKuXVjMXLDrEWDH/9jSWBsZ36xi7GDk5JARMJKaf3sIEYYtJ XLi3nq2LkYtDSGAJo8SOo5tYQRJCAguYJH5MrwKx2QTMJH53NDCD2CICUhLdbyYzQTTMZpQ4 0XOFHSTBKRAs8W3GRLAiYQFDiceNL8BsFgFVib9H54AN5RWwlHi5/TMzhC0o8WPyPRYQm1kg WmL3uZVQFylI7Dj7mhFimZPE491T2CBqxCUmPXjIPoFRYBaS9llI2mchKZvFyAFkG0pceu4L EZaX2P52DjOEbSBx4clkqFZtiWULXzMvYGRfxShaWpxcUJyUnmukV5yYW1yal66XnJ+7iRES 5sU7GF9ssDrEKMDBqMTD69F4NEiINbGsuDL3EKMEB7OSCG9cMVCINyWxsiq1KD++qDQntfgQ IxMHp1QDo4FA5T77+b7WsSsmCnjoqCyIZLy1nv1ahEHGz54LyhqLnjYVBIi0hvupOG/bfOVS ekTRlXcdClKrjx+SFDy5xLXf5My+e2XMq3SyZx8Pvjvn3JFTNxufvHxa6dulGHQsUvYS67Nt OSZyH5a/eOxwXdpo+SFGQ8nAf7tapjXzTDxX98l7qqymqxJLcUaioRZzUXEiAPVfz51RAgAA X-Virus-Checked: Checked by ClamAV on apache.org This is a multipart message in MIME format. ------=_NextPart_000_0167_01CF07A5.D89A1A90 Content-Type: text/plain; charset="iso-2022-jp" Content-Transfer-Encoding: 7bit Perhaps some of the data you require can be obtained with the mapred tool (here I’m using 2.2.0) e.g. htf@ivygerman:~/hadoop/bin$ ./mapred job Usage: CLI [-submit ] [-status ] [-counter ] [-kill ] [-set-priority ]. Valid values for priorities are: VERY_HIGH HIGH NORMAL LOW VERY_LOW [-events <#-of-events>] [-history ] [-list [all]] [-list-active-trackers] [-list-blacklisted-trackers] [-list-attempt-ids ]. Valid values for are REDUCE MAP. Valid values for are running, completed [-kill-task ] [-fail-task ] [-logs ] Generic options supported are -conf specify an application configuration file -D use value for given property -fs specify a namenode -jt specify a job tracker -files specify comma separated files to be copied to the map reduce cluster -libjars specify comma separated jar files to include in the classpath. -archives specify comma separated archives to be unarchived on the compute machines. The general command line syntax is bin/hadoop command [genericOptions] [commandOptions] htf@ivygerman:~/hadoop/bin$ ./mapred job -list 14/01/02 11:29:54 INFO client.RMProxy: Connecting to ResourceManager at ivy02.spa.sarc.sas/172.31.204.11:8032 Total jobs:1 JobId State StartTime UserName Queue Priority UsedContainers RsvdContainers UsedMem RsvdMem NeededMem AM info job_1388162829887_0050 RUNNING 1388678456812 htf root.htf NORMAL 300 20 307712M 20480M 328192M ivygerman-192-168-10-12.htfcluster.org:8088/proxy/application_1388162829887_ 0050/ htf@ivygerman-192-168-10-12:~/hadoop/bin$ From: Azuryy Yu [mailto:azuryyyu@gmail.com] Sent: Monday, December 30, 2013 10:42 PM To: user@hadoop.apache.org Subject: Re: get job data in command line in MRv2 Generally, MRv2 indicates Yarn. you can try: yarn application then there are full help lists. On Tue, Dec 31, 2013 at 12:32 PM, 小网客 wrote: ui or hadoop job command like：hadoop job -list ------------------ --------------------------------------------- BestWishes！ 小网客 Blog: http://snv.iteye.com/ Email:1134687142@qq.com ------------------ Original ------------------ From: "xeon";; Date: Tue, Dec 31, 2013 03:19 AM To: "user"; Subject: get job data in command line in MRv2 Hi, I would like to know if the MRv2 provide the following commands through the bash command line: - get the number of jobs running? - get the percentage of job completion of jobs? - get the number of jobs that are waiting to be submitted? -- Thanks, . ------=_NextPart_000_0167_01CF07A5.D89A1A90 Content-Type: text/html; charset="iso-2022-jp" Content-Transfer-Encoding: quoted-printable

Perhaps some of the data you require can be obtained with the mapred = tool (here I=1B$B!G=1B(Jm using 2.2.0)

 

e.g.

 

 

htf@ivygerman:~/hadoop/bin$ = ./mapred job

Usage: CLI <command> = <args>

      =   [-submit <job-file>]

      =   [-status <job-id>]

      =   [-counter <job-id> <group-name> = <counter-name>]

      =   [-kill <job-id>]

      =   [-set-priority <job-id> <priority>]. Valid values for = priorities are: VERY_HIGH HIGH NORMAL LOW VERY_LOW

      =   [-events <job-id> <from-event-#> = <#-of-events>]

      =   [-history <jobHistoryFile>]

      =   [-list [all]]<=1B(J/p>

      =   [-list-active-trackers]

      =   [-list-blacklisted-trackers]

      =   [-list-attempt-ids <job-id> <task-type> = <task-state>]. Valid values for <task-type> are REDUCE MAP. = Valid values for <task-state> are running, completed

      =   [-kill-task <task-attempt-id>]

      =   [-fail-task <task-attempt-id>]

      =   [-logs <job-id> <task-attempt-id>]

 

Generic options supported are

-conf <configuration = file>     specify an application configuration = file

-D = <property=3Dvalue>        &= nbsp;   use value for given property

-fs = <local|namenode:port>      specify a = namenode

-jt = <local|jobtracker:port>    specify a job = tracker

-files <comma separated list of = files>    specify comma separated files to be copied = to the map reduce cluster

<=1B(Jp class=3DMsoNormal = style=3D'text-autospace:none'>-libjars <comma separated list = of jars>    specify comma separated jar files to = include in the classpath.

-archives <comma separated list = of archives>    specify comma separated archives to be = unarchived on the compute machines.

 

The general command line syntax = is

bin/hadoop command [genericOptions] = [commandOptions]

 

htf@ivygerman:~/hadoop/bin$ = ./mapred job -list

14/01/02 11:29:54 INFO = client.RMProxy: Connecting to ResourceManager at = ivy02.spa.sarc.sas/172.31.204.11:8032

Total jobs:1

      =             = JobId      = State           = StartTime      = UserName           = Queue      = Priority       UsedContainers  = RsvdContainers  = UsedMem         = RsvdMem         = NeededMem         AM info

= job_1388162829887_0050    = RUNNING       = 1388678456812       =     htf        = root.htf        = NORMAL           &= nbsp;      = 300           &nbs= p;  20  = 307712M          = 20480M           = 328192M =1B(J     = ivygerman-192-168-10-12.htfcluster.org:8088/proxy/application_13881628298= 87_0050/

htf@ivygerman-192-168-10-12:~/hadoop= /bin$

 

 

 

From:= = Azuryy Yu [mailto:azuryyyu@gmail.com]
Sent: Monday, December = 30, 2013 10:42 PM
To: = user@hadoop.apache.org
Subject: Re: get job data in command = line in MRv2

 

Generally, MRv2 indicates Yarn. you can = try:

 

yarn application 

 

then there are full help = lists.

 

On Tue, Dec 31, 2013 at 12:32 PM, =1B$B>.f&5R=1B(J <smallnetvisitor@foxmail.com> = wrote:

ui or hadoop job command = like=1B$B!'=1B(Jhadoop job = -list

 

------------------<= /p>

-----------= ----------------------------------

BestWishes<= /span>=1B$B!*=1B(J=

=1B$B>.f&5R=1B(J=

=

 

 

 

------------------ Original -------------= -----

From: =  "xeon";<psdc1978@gmail.com>;

=

Date:  Tue, Dec 31, 2013 03:19 = AM

<=1B(Jb>To: =  "user"<user@hadoop.apache.org>; =

Subject: =  get job data in command = line in MRv2

 

Hi,

I would like to know if the = MRv2 provide the following commands through
the bash command = line:
- get the number of jobs running?
- get the percentage of = job completion of jobs?
- get the number of jobs that are waiting to = be submitted?

--
Thanks,

.

 

------=_NextPart_000_0167_01CF07A5.D89A1A90--