hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-807) When querying apps by queue, iterating over all apps is inefficient and limiting
Date Fri, 08 Aug 2014 08:11:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14090455#comment-14090455
] 

Wangda Tan commented on YARN-807:
---------------------------------

Hi Sandy, 
Thanks for your elaboration. As you said, I agree we need to go through scheduler according
to two capabilities you mentioned.
Maybe a possible way is saving completed app in leaf queue as you mentioned, I remember now
YARN will evict some apps when total number of apps exceeds a configuration number (like 10,000).
We should do such evicting for completed app in leaf queue as well.

> When querying apps by queue, iterating over all apps is inefficient and limiting 
> ---------------------------------------------------------------------------------
>
>                 Key: YARN-807
>                 URL: https://issues.apache.org/jira/browse/YARN-807
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.0.4-alpha
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>             Fix For: 2.3.0
>
>         Attachments: YARN-807-1.patch, YARN-807-2.patch, YARN-807-3.patch, YARN-807-4.patch,
YARN-807.patch
>
>
> The question "which apps are in queue x" can be asked via the RM REST APIs, through the
ClientRMService, and through the command line.  In all these cases, the question is answered
by scanning through every RMApp and filtering by the app's queue name.
> All schedulers maintain a mapping of queues to applications.  I think it would make more
sense to ask the schedulers which applications are in a given queue. This is what was done
in MR1. This would also have the advantage of allowing a parent queue to return all the applications
on leaf queues under it, and allow queue name aliases, as in the way that "root.default" and
"default" refer to the same queue in the fair scheduler.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message