hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1963) Support priorities across applications within the same queue
Date Wed, 19 Nov 2014 19:45:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14218381#comment-14218381

Wangda Tan commented on YARN-1963:

Thanks for reply,

bq. All we need a YarnClient implementation for taking this config and setting to ApplicationSubmissionContext.
( Something similar to queue name which this app is submitted to ).
Yes, that will be helpful, I wanna make sure they're not in YarnClient now? (including the
queue). I didn't see any related code in YarnClient

bq. The idea sounds good. The reason for specifying each label needed for a queue is because
admin can specify the labels applicable for a queue. With high priority, we may always end
up having default acceptance of lower priorities. How do you feel about having this as a range
Instead of having low-high range, I'd prefer highest + default priority. Admin can specify
highest priority for queue/user, and default priority for queue/user

bq. I have a use case scenario here. There are few applications running in a queue from 4
different users (sub...
I understood the use case here, but I think maybe an easier way is not change the definition
of user limit. Like having preemption mechanism to support higher priority applications take
resource from lower priority applications, etc. Divide user limit by priority will add extra
complexity both in implementation and configuration.

bq.  I suggest to add preemption within queue considering priority. ... +1. Already filed
a subjira for this.
The preemption I mentioned here is not YARN-2009, is to support the previous use case you
mentioned, we can keep user-limit as-is, and enforce higher priority application can get resource,
that should be possible :)


> Support priorities across applications within the same queue 
> -------------------------------------------------------------
>                 Key: YARN-1963
>                 URL: https://issues.apache.org/jira/browse/YARN-1963
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: api, resourcemanager
>            Reporter: Arun C Murthy
>            Assignee: Sunil G
>         Attachments: YARN Application Priorities Design.pdf
> It will be very useful to support priorities among applications within the same queue,
particularly in production scenarios. It allows for finer-grained controls without having
to force admins to create a multitude of queues, plus allows existing applications to continue
using existing queues which are usually part of institutional memory.

This message was sent by Atlassian JIRA

View raw message