hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rohith Sharma K S (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5378) Accommodate app-id->cluster mapping
Date Tue, 17 Jan 2017 07:04:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-5378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15825566#comment-15825566

Rohith Sharma K S commented on YARN-5378:

bq. For any UI we'll have to determine what we do if there is a collision between app_ids
on multiple clusters. In that case the user would have to be presented with a choice. In the
normal case we can simply pick the first and only cluster and move on.
What I see UI admin is, ClusterId as landing page or left panel listed with clusterId's with
listing first clusterId flows as landing page. This is easier for Admin to manage multi cluster.
Admin/User need to choose clusterId and drill down to the applications. But when clusterId's
are unknown, it is hard to find which one to choose. So, apart from listing down clusterId's
given applicationId i.e YARN-6095,  It would be better if  there is an API to list down all
the ClusterId's. Thoughts?

> Accommodate app-id->cluster mapping
> -----------------------------------
>                 Key: YARN-5378
>                 URL: https://issues.apache.org/jira/browse/YARN-5378
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Joep Rottinghuis
>            Assignee: Sangjin Lee
>              Labels: yarn-5355-merge-blocker
>         Attachments: YARN-5378-YARN-5355.01.patch, YARN-5378-YARN-5355.02.patch, YARN-5378-YARN-5355.03.patch
> In discussion with [~sjlee0], [~vrushalic], [~subru], and [~curino] a use-case came up
to be able to map from application-id to cluster-id in context of federation for Yarn.
> What happens is that a "random" cluster in the federation is asked to generate an app-id
and then potentially a different cluster can be the "home" cluster for the AM. Furthermore,
tasks can then run in yet other clusters.
> In order to be able to pull up the logical home cluster on which the application ran,
there needs to be a mapping from application-id to cluster-id. This mapping is available in
the federated Yarn case only during the active live of the application.
> A similar situation is common in our larger production environment. Somebody will complain
about a slow job, some failure or whatever. If we're lucky we have an application-id. When
we ask the user which cluster they ran on, they'll typically answer with the machine from
where they launched the job (many users are unaware of the underlying physical clusters).
This leaves us to spelunk through various RM ui's to find a matching epoch in the application

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message