hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-5378) Accomodate app-id->cluster mapping
Date Thu, 14 Jul 2016 07:28:20 GMT
Joep Rottinghuis created YARN-5378:

             Summary: Accomodate app-id->cluster mapping
                 Key: YARN-5378
                 URL: https://issues.apache.org/jira/browse/YARN-5378
             Project: Hadoop YARN
          Issue Type: Sub-task
          Components: timelineserver
            Reporter: Joep Rottinghuis
            Assignee: Joep Rottinghuis

In discussion with [~sjlee0], [~vrushalic], [~subru], and [~curino] a use-case came up to
be able to map from application-id to cluster-id in context of federation for Yarn.
What happens is that a "random" cluster in the federation is asked to generate an app-id and
then potentially a different cluster can be the "home" cluster for the AM. Furthermore, tasks
can then run in yet other clusters.
In order to be able to pull up the logical home cluster on which the application ran, there
needs to be a mapping from application-id to cluster-id. This mapping is available in the
federated Yarn case only during the active live of the application.

A similar situation is common in our larger production environment. Somebody will complain
about a slow job, some failure or whatever. If we're lucky we have an application-id. When
we ask the user which cluster they ran on, they'll typically answer with the machine from
where they launched the job (many users are unaware of the underlying physical clusters).
This leaves us to spelunk through various RM ui's to find a matching epoch in the application

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message