Return-Path: X-Original-To: apmail-hadoop-common-user-archive@www.apache.org Delivered-To: apmail-hadoop-common-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A8138108F0 for ; Fri, 30 May 2014 09:50:08 +0000 (UTC) Received: (qmail 64058 invoked by uid 500); 30 May 2014 09:50:04 -0000 Delivered-To: apmail-hadoop-common-user-archive@hadoop.apache.org Received: (qmail 63837 invoked by uid 500); 30 May 2014 09:50:04 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 63822 invoked by uid 99); 30 May 2014 09:50:04 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 May 2014 09:50:04 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of rohithsharmaks@huawei.com designates 119.145.14.65 as permitted sender) Received: from [119.145.14.65] (HELO szxga02-in.huawei.com) (119.145.14.65) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 30 May 2014 09:49:59 +0000 Received: from 172.24.2.119 (EHLO szxeml213-edg.china.huawei.com) ([172.24.2.119]) by szxrg02-dlp.huawei.com (MOS 4.3.7-GA FastPath queued) with ESMTP id BUM19810; Fri, 30 May 2014 17:49:30 +0800 (CST) Received: from SZXEML405-HUB.china.huawei.com (10.82.67.60) by szxeml213-edg.china.huawei.com (172.24.2.30) with Microsoft SMTP Server (TLS) id 14.3.158.1; Fri, 30 May 2014 17:49:26 +0800 Received: from SZXEML512-MBS.china.huawei.com ([169.254.8.162]) by szxeml405-hub.china.huawei.com ([::1]) with mapi id 14.03.0158.001; Fri, 30 May 2014 17:49:24 +0800 From: Rohith Sharma K S To: "user@hadoop.apache.org" Subject: RE: change yarn application priority Thread-Topic: change yarn application priority Thread-Index: Ac97zUkt1ueeaASkSaWpGKoCXaXv8wAGexaw Date: Fri, 30 May 2014 09:49:24 +0000 Message-ID: <0EE80F6F7A98A64EBD18F2BE839C9115675D836A@szxeml512-mbs.china.huawei.com> References: <53DC189E5FAEFA43BFA1BC02431031DB740A45BB7A@WECTMLBOX.winbond.com.tw> In-Reply-To: <53DC189E5FAEFA43BFA1BC02431031DB740A45BB7A@WECTMLBOX.winbond.com.tw> Accept-Language: en-US, zh-CN Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.18.168.138] Content-Type: multipart/alternative; boundary="_000_0EE80F6F7A98A64EBD18F2BE839C9115675D836Aszxeml512mbschi_" MIME-Version: 1.0 X-CFilter-Loop: Reflected X-Virus-Checked: Checked by ClamAV on apache.org --_000_0EE80F6F7A98A64EBD18F2BE839C9115675D836Aszxeml512mbschi_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Currently there is no provision for changing application priority within= the same queue. Follow the Jira https://issues.apache.org/jira/i#browse/Y= ARN-1963 for this new feature. One way you can achieve by using enabling scheduler monitors for CapacitySc= hedulers. Steps to be follow is 1. Configure 2 queues, follow http://hadoop.apache.org/docs/r2.3.0/ha= doop-yarn/hadoop-yarn-site/CapacityScheduler.html 2. Enable scheduler monitor yarn.resourcemanager.scheduler.monitor.enable =3D true One job you submit to queue 1 which run 2hours. Another job you submit queu= e 2. Hope this will help you. Thanks & Regards Rohith Sharma K S This e-mail and its attachments contain confidential information from HUAWE= I, which is intended only for the person or entity whose address is listed above. An= y use of the information contained herein in any way (including, but not limited to, tot= al or partial disclosure, reproduction, or dissemination) by persons other than the inten= ded recipient(s) is prohibited. If you receive this e-mail in error, please not= ify the sender by phone or email immediately and delete it! From: Henry Hung [mailto:YTHung1@winbond.com] Sent: 30 May 2014 11:53 To: user@hadoop.apache.org Subject: change yarn application priority HI All, I have an application that consumes all of nodemanager capacity (30 Map and= 1 Reducer) and will need 4 hours to finish. Let's say I need to run another application that will be quicker to finish = (30 minutes) and only need 1 Map and 1 Reducer. If I just execute the new application, it will be in queue waiting for the = 1st application to finish. Is there a way to change the 2nd application priority to higher than the 1s= t and let resourcemanager immediately execute the 2nd application? I'm using Hadoop-2.2.0. Best regards, Henry ________________________________ The privileged confidential information contained in this email is intended= for use only by the addressees as indicated by the original sender of this= email. If you are not the addressee indicated in this email or are not res= ponsible for delivery of the email to such a person, please kindly reply to= the sender indicating this fact and delete all copies of it from your comp= uter and network server immediately. Your cooperation is highly appreciated= . It is advised that any unauthorized use of confidential information of Wi= nbond is strictly prohibited; and any information in this email irrelevant = to the official business of Winbond shall be deemed as neither given nor en= dorsed by Winbond. --_000_0EE80F6F7A98A64EBD18F2BE839C9115675D836Aszxeml512mbschi_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi

 

   Currently= there is no provision for changing application priority within the same qu= eue.  Follow the Jira https://issue= s.apache.org/jira/i#browse/YARN-1963 for this new feature.

 

One way you can achiev= e by using enabling scheduler monitors for CapacitySchedulers.

Steps to be follow is =

1.&= nbsp;      Configure 2 qu= eues, follow http://hadoop.apache.org/docs/r2.3.0/hadoo= p-yarn/hadoop-yarn-site/CapacityScheduler.html

2.&= nbsp;      Enable schedul= er monitor

yarn.resourcema= nager.scheduler.monitor.enable =3D true

 

One job you submit to = queue 1 which run 2hours. Another job you submit queue 2.=

 

Hope this will help yo= u.

 

Thanks & Regards

Rohith Sharma K S=

 

 

This e-mail and its attachments= contain confidential information from HUAWEI, which
is intended only for the person or entity whose address is listed above. An= y use of the
information contained herein in any way (including, but not limited to, tot= al or partial
disclosure, reproduction, or dissemination) by persons other than the inten= ded
recipient(s) is prohibited. If you receive this e-mail in error, please not= ify the sender by
phone or email immediately and delete it!

 

From: Henry Hu= ng [mailto:YTHung1@winbond.com]
Sent: 30 May 2014 11:53
To: user@hadoop.apache.org
Subject: change yarn application priority

 

HI All,

 

I have an application that consumes all of nodemanag= er capacity (30 Map and 1 Reducer) and will need 4 hours to finish.

Let’s say I need to run another application th= at will be quicker to finish (30 minutes) and only need 1 Map and 1 Reducer= .

If I just execute the new application, it will be in= queue waiting for the 1st application to finish.

Is there a way to change the 2nd applicat= ion priority to higher than the 1st and let resourcemanager immediately exe= cute the 2nd application?

 

I’m using Hadoop-2.2.0.

 

Best regards,

Henry

 

The privileged confidential inf= ormation contained in this email is intended for use only by the addressees= as indicated by the original sender of this email. If you are not the addressee indicated in this email or are not responsible for d= elivery of the email to such a person, please kindly reply to the sender in= dicating this fact and delete all copies of it from your computer and netwo= rk server immediately. Your cooperation is highly appreciated. It is advised that any unauthorized use of confiden= tial information of Winbond is strictly prohibited; and any information in = this email irrelevant to the official business of Winbond shall be deemed a= s neither given nor endorsed by Winbond.

--_000_0EE80F6F7A98A64EBD18F2BE839C9115675D836Aszxeml512mbschi_--