hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1001) YARN should provide per application-type and state statistics
Date Tue, 17 Sep 2013 16:21:53 GMT

    [ https://issues.apache.org/jira/browse/YARN-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769645#comment-13769645
] 

Xuan Gong commented on YARN-1001:
---------------------------------

bq. RMContext store all the applications in one map. This data structure is not friendly to
GetApps and GetAppStatistics, which need to go through the whole collection. Given some regular
query criteria, such as states and applicationTypes, it would be good if we can somehow index
the apps according to these criteria. Therefore, we can speed up the two APIs, maybe also
some other operations on the app collection of RMContext.

I think that we are fine to use ConcurrentMap. Because most of time, we are just using add/remove.
The situation that need to go through the whole collection to find a specific value mostly
exist in webService and CLIs, and those webService request and client requests are very high
frequency (at least not as frequent as we are using add/remove). So, we are fine to just use
ConcurrentMap data structure.

bq. Now web API ignores the case when matching the applicationType. IMHO, ApplicationClientProtocol
should have the same behavior. Moreover, I think it's good to define criteria of naming an
applicationType, for example, letter and number only, case insensitive and etc.

About the case insensitive, I think that happens mostly when the users do the input. Since
we have already take care of those case insensitive in ApplicationCLI methods or WebService
methods, I do not think ApplicationClientProtocol (ClientRMService) need to consider this.
So, for ApplicationClientProtocol, everything can be capital, for the users input, they can
be either capital or lower case. When we do actually comparison(in applicationCLI methods
or WebService methods), we can convert them to either capital or lower case. That is what
we are doing currently, and I think it is enough.

                
> YARN should provide per application-type and state statistics
> -------------------------------------------------------------
>
>                 Key: YARN-1001
>                 URL: https://issues.apache.org/jira/browse/YARN-1001
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api
>    Affects Versions: 2.1.0-beta
>            Reporter: Srimanth Gunturi
>            Assignee: Zhijie Shen
>            Priority: Blocker
>             Fix For: 2.1.1-beta
>
>         Attachments: YARN-1001.1.patch, YARN-1001.2.patch, YARN-1001.3.patch, YARN-1001.4.patch,
YARN-1001.5.patch, YARN-1001.6.patch, YARN-1001.7.patch
>
>
> In Ambari we plan to show for MR2 the number of applications finished, running, waiting,
etc. It would be efficient if YARN could provide per application-type and state aggregated
counts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message