hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2513) Host framework UIs in YARN for use with the ATS
Date Sun, 14 Sep 2014 05:44:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14133096#comment-14133096

Zhijie Shen commented on YARN-2513:

[~jeagles], thanks for working on this feature, which seems to be interesting. The benefit
I can see with integrating the per-framework UI widget into the timeline server is to relieve
the framework from deploying its own web server, and to prevent additional data move from
the timeline server to the framework web server, and finally to the user.

To enable the web plugin, so far I can think of the following things that it's good to address.

1. Security: there're three potential issues. a) Do we want to use Hadoop http authentication
to protect the pre-framework web plugin? b) We don't page-level or url matching access authorization,
such that, for example, UserA can only access the web pages of it's authorized web plugin.
c) Currently, everything inside timeline server belongs to YARN, such that we don't limit
its access to the resources internally. Web plugin is hosted by the timeline server on behalf
of the framework, and it should only have the access to the resources granted to it. For example,
Tez webUI should only have access to the tez metrics in the timeline store.

2. Isolation: for some reason, the web plugin is going to crash, we should make sure it is
not going to affect other components in the timeline server and other web plugins. Moreover,
if multiple web plugins are hosted in the timeline server, we need to take care of their competing
for the web server resources, preventing starvation.

3. Scalability: now everything is hosted in a single web server container. Hosting framework
UIs will drive more traffic to the timeline server. We may want to scale up web server instances
to handle users' requests. In addition, it's good think of whether we want distribute the
workload according the functions: some of them serves raw REST APIs, others serve web UI of
framework 1, 2, 3 and the remaining ones serve web UI of framework 4, 5, 6.

> Host framework UIs in YARN for use with the ATS
> -----------------------------------------------
>                 Key: YARN-2513
>                 URL: https://issues.apache.org/jira/browse/YARN-2513
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Jonathan Eagles
>            Assignee: Jonathan Eagles
>         Attachments: YARN-2513-v1.patch
> Allow for pluggable UIs as described by TEZ-8. Yarn can provide the infrastructure to
host java script and possible java UIs.

This message was sent by Atlassian JIRA

View raw message