Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 23 Jan 2017 13:15:26 +0000 (UTC)
From: "Varun Saxena (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13035656.1484684027000.79261.1485177326750@Atlassian.JIRA>
In-Reply-To: <JIRA.13035656.1484684027000@Atlassian.JIRA>
References: <JIRA.13035656.1484684027000@Atlassian.JIRA> <JIRA.13035656.1484684027552@jira-lw-us.apache.org>
Subject: [jira] [Comment Edited] (YARN-6105) Support for new REST end point
 /clusterids
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 23 Jan 2017 13:15:38 -0000


    [ https://issues.apache.org/jira/browse/YARN-6105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834435#comment-15834435 ] 

Varun Saxena edited comment on YARN-6105 at 1/23/17 1:15 PM:
-------------------------------------------------------------

bq. I discussed with cloud team today, constant clusterId or storing it separately as info does not work for them.
Hmm...Trying to understand the issue better. What issues do they face if they publish entities with a constant cluster ID in the cloud? Is there any advantage in differentiating between different ephemeral clusters, and that too as part of the context(i.e. via YARN tags) ? In a cloud scenario, does the user even care about the different cluster IDs' each time his job is run? Wouldn't cluster information be provided by the Cloud Manager or we plan to have Cloud Manager get this info too from ATS? Agree that each cluster is unique, but if cluster ID is dynamically generated at runtime as clusters come up and go away, we need to see how much sense this cluster ID would make for end user. 
Anyways, assuming that for cloud scenario, a constant cluster is not to be given,  if we want a list of all cluster IDs', I guess we would need it at real time. In that case we cannot have a periodic job finding out unique cluster IDs' and adding them to the new table as suggested above. Correct ?

I am fine with having a /clusterids per say as I mentioned on the other JIRA, but for cloud use case, wouldn't this list become quite large? And as it becomes quite large, we need to see how useful it is.

We can probably provide some API to register cluster if the use case for cloud is strong enough and there is no feasible alternative. Thoughts ?


was (Author: varun_saxena):
bq. I discussed with cloud team today, constant clusterId or storing it separately as info does not work for them.
Hmm...Trying to understand the issue better. What issues do they face if they publish entities with a constant cluster ID across the cloud? Is there any advantage in differentiating between different ephemeral clusters, and that too as part of the context(i.e. via YARN tags) ? In a cloud scenario, does the user even care about the different cluster IDs' each time his job is run? Wouldn't cluster information be provided by the Cloud Manager or we plan to have Cloud Manager get this info too from ATS ?
Anyways, assuming that for cloud scenario, a constant cluster is not to be given,  if we want a list of all cluster IDs', I guess we would need it at real time. In that case we cannot have a periodic job finding out unique cluster IDs' and adding them to the new table as suggested above. Correct ?

I am fine with having a /clusterids per say as I mentioned on the other JIRA, but for cloud use case, wouldn't this list become quite large? And as it becomes quite large, we need to see how useful it is.

> Support for new REST end point /clusterids
> ------------------------------------------
>
>                 Key: YARN-6105
>                 URL: https://issues.apache.org/jira/browse/YARN-6105
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Rohith Sharma K S
>
> As discussed in YARN-5378 and YARN-6095, it is required to have */clusterids* that returns list of clusterids that back end has is useful. 
> Use case : In cloud, clusters are arbitrarily spin up and destroyed. Each cluster has its own clusterId which UI never knows about it. To all those newly spin up cluster, same ATS server has been used. And sam web UI has been used. Admin can select the clusterId and navigate to any pages. So, it is worth to list ClusterId's from ATS


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org