hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7254) UI and metrics changes related to absolute resource configuration
Date Wed, 11 Oct 2017 01:06:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16199651#comment-16199651
] 

Wangda Tan commented on YARN-7254:
----------------------------------

[~sunilg],

Took a quick look at the patch and tried in my local cluster:

In general, it's better to make sure for all user-facing places ((REST)API/UI/CLI), include
the information of is the resource is a effective or configured. And on UI/API/REST-API, it's
better to include both.

For example, on REST API (should we add {{-mb}} suffix to memory as well? And change vCores
to vcores):
{code}
  <minResource>
    <memory>6000</memory>
    <vCores>5</vCores>
  </minResource>
  <maxResource>
    <memory>6000</memory>
    <vCores>12</vCores>
  </maxResource>
{code}

And On queue page, it says: 
{code}
Configured Capacity:  <memory:6000, vCores:5> (100.0%)
{code}
However it is effective capacity (100.0% is calculated as well).

Since we only allow admin to configure either absolute or percentage, we can make the default
to -1 to indicate it is not set by admin.

Others:
- QueueConfigurations not set by anybody, did you forget to set it or we should move this
to a separate patch? Like I mentioned above, it's better to include both effective/configured
resources in the java API.

*Tests on a pusedo cluster:*

Configured a single queue in c-s.xml: Queue.capacity = (8192, 8)
Start cluster with one node: (8192, 8). Information shown on UI looks correct.
Updated node's resource to: (6000, 12), Information shown on UI:

{code}
- Configured Capacity: <memory:6000, vCores:5> (100.0%) (Should this be vcore=8?).
- Max capacity. <memory:6000, vCores:12> (100.0%)
{code}





> UI and metrics changes related to absolute resource configuration
> -----------------------------------------------------------------
>
>                 Key: YARN-7254
>                 URL: https://issues.apache.org/jira/browse/YARN-7254
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacity scheduler
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-7254.001.patch, YARN-7254.002.patch, YARN-7254.YARN-5881.002.patch
>
>
> Impact on UI and metrics related to absolute resource configuration on CS



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message