hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7215) REST API to list all deployed services by the same user
Date Wed, 20 Sep 2017 06:26:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172816#comment-16172816

Eric Yang commented on YARN-7215:

Slider list was not robust.  It take multiple seconds to respond to the query.  If there are
several hundred users using a UI to manage user's own application.  The retrieval of information
should have low range of millisecond response time.  When application data are persisted in
the same metastore with index and search capability, it will be easier to use the same storage
mechanism to build application catalog.  Although it is easy to build a view of YARN deployed
applications base on computing metadata stored on HDFS and ZooKeeper.  However, those services
are not optimized for serving web application REST API.  Let's take one step further on reducing
too many small file problem on HDFS and too big z-node on ZooKeeper in consideration of the
design.  This will help to steer developers toward good design pattern.  I am open to suggestion
to list yarn applications which can survive ResourceManager restart.

> REST API to list all deployed services by the same user
> -------------------------------------------------------
>                 Key: YARN-7215
>                 URL: https://issues.apache.org/jira/browse/YARN-7215
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, applications
>            Reporter: Eric Yang
>            Assignee: Eric Yang
> In Slider, it is possible to list deployed applications from the same user by using:
> {code}
> slider list
> {code}
> This API can help UI to display application and services deployed by the same user.
> Apiserver does not have ability to list all applications/services at this time.  This
API requires fast response to list all applications because it is a common UI operation. 
ApiServer deployed applications persist configuration in HDFS similar to slider, but using
directory listing to display deployed application might cost too much overhead to namenode.
 We may want to use alternative storage mechanism to cache deployed application configuration
to accelerate the response time of list deployed applications.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message