hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ryza <sandy.r...@cloudera.com>
Subject Re: How to really add priority for job and queue?
Date Fri, 01 Aug 2014 00:33:36 GMT
Hi Cui,

Fair Scheduler queues can be given priorities via weights.  The Fair
Scheduler documentation discusses how to configure these -
http://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/FairScheduler.html.
 Allowing applications to have priorities is planned, but I don't remember
the JIRA off of the top of my head.

Tying queues to particular nodes is currently being discussed on YARN-796.
 There are differing opinions on whether we should include this feature.

-Sandy


On Thu, Jul 31, 2014 at 5:26 PM, Cui Liqiang <cui.liqiang@gmail.com> wrote:

> Hi,
>
> I am using Yarn and facing some custom requirements which seem not
> supported yet:
> Currently the getPriority() implementation in FSQueue and AppScheduler are
> all hard coded to be 1. We’d like them to reflect the real priority, say,
> from configuration file(or allocation file?).
> We’d like to restrict container(or node?) assignments of some queues, e.g.
> we want queueA only assigned to node with ip 10.10.10.10
>
> From my understanding, if I want to implement these requirements, I have
> to modify some codes in Yarn and probably do some extension as well. Thus I
> may have troubles in resolving conflicts when upgrading Yarn version. So I
> am wondering:
> Is there any way to achieve those two requirements without change the code?
> If not, does Yarn have any plans to implement these?
> If not, does these requirements sound general enough so that I can
> finished it by ourselves and try to merge it back to the trunk?
>
> I am pretty new to Yarn, so If what I am asking sounds silly, please let
> me know and give the direction.
>
>
> Thanks.
>
> --
> Cui Liqiang
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message