hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3273) Improve web UI to facilitate scheduling analysis and debugging
Date Sat, 14 Mar 2015 04:07:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-3273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361586#comment-14361586
] 

Jian He commented on YARN-3273:
-------------------------------

thanks Rohith ! overall looks great !
- synchronization of userResourceLimit - I think we can use volatile keyword for the userResourceLimit
and do not need the synchronized keyword
{code}
public Resource getUserResourceLimit() {
  return userResourceLimit;
}

public synchronized void setUserResourceLimit(Resource userResourceLimit) {
  this.userResourceLimit
= userResourceLimit;
}
{code}
- SchedulerCommonInfo -> SchedulerInfo
- I think it’s fine to store below info with Resource type. i.e. “private Resource minAllocResource”;
similarly on the UI, we can expose as minimum allocation resource
{code}
protected long minAllocMemory;
protected long maxAllocMemory;
protected long minAllocVirtualCores;
protected
long maxAllocVirtualCores;
{code}
- just for better code readability, we may use a variable to store the entry.getValue
{code}
     usersToReturn.add(new UserInfo(entry.getKey(), Resources.clone(entry
          .getValue().getUsed()), entry.getValue().getActiveApplications(),
          entry.getValue().getPendingApplications(), Resources.clone(entry
              .getValue().getConsumedAMResources()), Resources.clone(entry
              .getValue().getUserResourceLimit())));
{code}
- the headroom rendering should be inside the "webUiType.equals(YarnWebParams.RM_WEB_UI)"
check.
{code}// TODO Need to get HeadRoom from scheduler and render it web ui {code}
bq.  I think headroom can be in RMAppAttemptMetric and render only if attempt is running.
sounds good to me.

> Improve web UI to facilitate scheduling analysis and debugging
> --------------------------------------------------------------
>
>                 Key: YARN-3273
>                 URL: https://issues.apache.org/jira/browse/YARN-3273
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Jian He
>            Assignee: Rohith
>         Attachments: 0001-YARN-3273-v1.patch, 0001-YARN-3273-v2.patch, YARN-3273-am-resource-used-AND-User-limit-v2.PNG,
YARN-3273-am-resource-used-AND-User-limit.PNG, YARN-3273-application-headroom-v2.PNG, YARN-3273-application-headroom.PNG
>
>
> Job may be stuck for reasons such as:
> - hitting queue capacity 
> - hitting user-limit, 
> - hitting AM-resource-percentage 
> The  first queueCapacity is already shown on the UI.
> We may surface things like:
> - what is user's current usage and user-limit; 
> - what is the AM resource usage and limit;
> - what is the application's current HeadRoom;
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message