hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2408) Resource Request REST API for YARN
Date Wed, 13 May 2015 21:56:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14542804#comment-14542804
] 

Wangda Tan commented on YARN-2408:
----------------------------------

[~rdelvalle],
Thanks for updating, I took at look at your latest patch, some overall comments:

1) About what the APIs should look like:
Basically we have two choices, one is to keep the original ResourceRequests structure in app,
which is:
- Priorities under an app
- ResourceRequests grouped by resourceName

And we should support return applications satisfy given criteria only, such as a given queue,
app-id, user, etc. We can support some of them with this patch, but return all apps in the
cluster is not useful/efficiency enough.

The other choice is aggregate ResourceRequests, like what you have done in your patch. Keep
less information and also less data needs to be tranfered from RM. I prefer the latter one.

2) How to get aggregate ResourceRequests, I suggest you can take a look at SchedulerApplicationAttempt.getAppAttemptResourceUsage,
which should contains what you needed. It is updated when you updates application's pending
resource. Also, I suggest to add pending resource-by-label as well since we already have YARN-796.

Thoughts?

> Resource Request REST API for YARN
> ----------------------------------
>
>                 Key: YARN-2408
>                 URL: https://issues.apache.org/jira/browse/YARN-2408
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: webapp
>            Reporter: Renan DelValle
>              Labels: features
>         Attachments: YARN-2408-6.patch
>
>
> I’m proposing a new REST API for YARN which exposes a snapshot of the Resource Requests
that exist inside of the Scheduler. My motivation behind this new feature is to allow external
software to monitor the amount of resources being requested to gain more insightful information
into cluster usage than is already provided. The API can also be used by external software
to detect a starved application and alert the appropriate users and/or sys admin so that the
problem may be remedied.
> Here is the proposed API (a JSON counterpart is also available):
> {code:xml}
> <resourceRequests>
>   <MB>7680</MB>
>   <VCores>7</VCores>
>   <appMaster>
>     <applicationId>application_1412191664217_0001</applicationId>
>     <applicationAttemptId>appattempt_1412191664217_0001_000001</applicationAttemptId>
>     <queueName>default</queueName>
>     <totalMB>6144</totalMB>
>     <totalVCores>6</totalVCores>
>     <numResourceRequests>3</numResourceRequests>
>     <requests>
>       <request>
>         <MB>1024</MB>
>         <VCores>1</VCores>
>         <numContainers>6</numContainers>
>         <relaxLocality>true</relaxLocality>
>         <priority>20</priority>
>         <resourceNames>
>           <resourceName>localMachine</resourceName>
>           <resourceName>/default-rack</resourceName>
>           <resourceName>*</resourceName>
>         </resourceNames>
>       </request>
>     </requests>
>   </appMaster>
>   <appMaster>
>   ...
>   </appMaster>
> </resourceRequests>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message