hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohith Sharma K S <rohithsharm...@huawei.com>
Subject RE: change yarn application priority
Date Fri, 30 May 2014 09:49:24 GMT

   Currently there is no provision for changing application priority within the same queue.
 Follow the Jira https://issues.apache.org/jira/i#browse/YARN-1963 for this new feature.

One way you can achieve by using enabling scheduler monitors for CapacitySchedulers.
Steps to be follow is

1.       Configure 2 queues, follow http://hadoop.apache.org/docs/r2.3.0/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

2.       Enable scheduler monitor

yarn.resourcemanager.scheduler.monitor.enable = true

One job you submit to queue 1 which run 2hours. Another job you submit queue 2.

Hope this will help you.

Thanks & Regards
Rohith Sharma K S

This e-mail and its attachments contain confidential information from HUAWEI, which
is intended only for the person or entity whose address is listed above. Any use of the
information contained herein in any way (including, but not limited to, total or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
phone or email immediately and delete it!

From: Henry Hung [mailto:YTHung1@winbond.com]
Sent: 30 May 2014 11:53
To: user@hadoop.apache.org
Subject: change yarn application priority

HI All,

I have an application that consumes all of nodemanager capacity (30 Map and 1 Reducer) and
will need 4 hours to finish.
Let's say I need to run another application that will be quicker to finish (30 minutes) and
only need 1 Map and 1 Reducer.
If I just execute the new application, it will be in queue waiting for the 1st application
to finish.
Is there a way to change the 2nd application priority to higher than the 1st and let resourcemanager
immediately execute the 2nd application?

I'm using Hadoop-2.2.0.

Best regards,

The privileged confidential information contained in this email is intended for use only by
the addressees as indicated by the original sender of this email. If you are not the addressee
indicated in this email or are not responsible for delivery of the email to such a person,
please kindly reply to the sender indicating this fact and delete all copies of it from your
computer and network server immediately. Your cooperation is highly appreciated. It is advised
that any unauthorized use of confidential information of Winbond is strictly prohibited; and
any information in this email irrelevant to the official business of Winbond shall be deemed
as neither given nor endorsed by Winbond.

View raw message