hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-451) Add more metrics to RM page
Date Tue, 15 Oct 2013 01:01:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13794737#comment-13794737
] 

Joep Rottinghuis commented on YARN-451:
---------------------------------------

[~jlowe] current allocation can be seen from the scheduler page.
While solving the "which app is the most starved" app is also an interesting use-case I think
that showing only what the current assignement is (and not what the estimated total ask will
be) does not solve the use case we're trying to tackle.
Today we again had an issue where a (newby) user launched 20 30K mapper jobs. While each job
got only few resources, the issue was not identifying that his jobs were all starved, but
more that his jobs were going to not finish within any reasonable time and would demand over
time an unreasonable amount of resources.

On a Hadoop 1.0 equivalent cluster one can quickly sort by # mappers /. # reducers.
We need the equivalent on Yarn where we can click on the estimated #MB that this app will
be asking for in total and/or the estimated total # cores.

> Add more metrics to RM page
> ---------------------------
>
>                 Key: YARN-451
>                 URL: https://issues.apache.org/jira/browse/YARN-451
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.0.3-alpha
>            Reporter: Lohit Vijayarenu
>            Assignee: Sangjin Lee
>            Priority: Blocker
>         Attachments: in_progress_2x.png, yarn-451-trunk-20130916.1.patch
>
>
> ResourceManager webUI shows list of RUNNING applications, but it does not tell which
applications are requesting more resource compared to others. With cluster running hundreds
of applications at once it would be useful to have some kind of metric to show high-resource
usage applications vs low-resource usage ones. At the minimum showing number of containers
is good option.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message