hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lohit Vijayarenu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-451) Add more metrics to RM page
Date Tue, 09 Apr 2013 22:28:17 GMT

    [ https://issues.apache.org/jira/browse/YARN-451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13627203#comment-13627203
] 

Lohit Vijayarenu commented on YARN-451:
---------------------------------------

Tried to see if adding total number of Containers was trivial change, but looks like there
is no notion of application max resource available to resourcemangaer. This might be the reason
why RM page did not have the information. Looking into RMAppImpl shows that this kind of information
is not passed either from Client/AM to RM during application initialization. 

Something close to notion of job weight I could see was Resource demand, but that seems to
be change based on how an application request containers. For example FairScheduler seem to
recalculate fairshare based on how much resource demand is passed by applications. 

One option I can think of is to add an additional field in protobuf which specifies what is
total number of containers/resource an application might use. This would be optional field
which can be used only by MapReduce for now and Client can set this value based on number
of mappers/reducers. I am not sure if this is the right approach, any other simpler ideas
people can suggest?
                
> Add more metrics to RM page
> ---------------------------
>
>                 Key: YARN-451
>                 URL: https://issues.apache.org/jira/browse/YARN-451
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.0.3-alpha
>            Reporter: Lohit Vijayarenu
>            Priority: Minor
>
> ResourceManager webUI shows list of RUNNING applications, but it does not tell which
applications are requesting more resource compared to others. With cluster running hundreds
of applications at once it would be useful to have some kind of metric to show high-resource
usage applications vs low-resource usage ones. At the minimum showing number of containers
is good option.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message