hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-2786) Create yarn cluster CLI to enable list node labels collection
Date Wed, 12 Nov 2014 17:40:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14208300#comment-14208300
] 

Allen Wittenauer edited comment on YARN-2786 at 11/12/14 5:40 PM:
------------------------------------------------------------------

bq. By the time I started on HOD, those experiments were presumably over, so I never heard
about them.

I'm not surprised... otherwise you'd realize what a mess having a state file for this info
causes. Every single time a node gets added, every single time a node gets modified, every
single time a node gets deleted, this state file is going to have to be verified.  This is
in addition to the already existing includex2, excludex2, and slaves files.

bq.Further, you don't want a central list of valid labels which I disagree with. 

The part that no one has had a reasonable explanation for is where exactly these invalid labels
are coming from.  If the operations teams are the ones setting up the cluster, shouldn't they
know what labels they want to use?  The only conclusion I can come up with is Ambari or Cloudera
Manager.  So, yes, I can see why people would need protection against them.

bq. Others like me can turn it on to avoid random-labels-bubbling-up-in-my-cluster madness.

You seem to be confusing 'random' and 'ephemeral'.  

The whole reason I'm against adding this change is that it just makes the state file situation
that much worse.  Now instead of being able to control the labels either via code or configuration
management, we're going to trust humans to type things in at the command line. Essentially:
adding this feature *creates* the invalid label problem you are hoping to avoid!  

So in summary, again, I'm completely and totally against the ability to add labels from the
command line.  It's a big flaw in an implementation full of flaws.


was (Author: aw):
bq. By the time I started on HOD, those experiments were presumably over, so I never heard
about them.

I'm not surprised... otherwise you'd realize what a mess having a state file for this info
causes. Every single time a node gets added, every single time a node gets modified, every
single time a node gets deleted, this state file is going to have to verified.  This is in
addition to the already existing includex2, excludex2, and slaves files.

bq.Further, you don't want a central list of valid labels which I disagree with. 

The part that no one has had a reasonable explanation for is where exactly these invalid labels
are coming from.  If the operations teams are the ones setting up the cluster, shouldn't they
know what labels they want to use?  The only conclusion I can come up with is Ambari or Cloudera
Manager.  So, yes, I can see why people would need protection against them.

bq. Others like me can turn it on to avoid random-labels-bubbling-up-in-my-cluster madness.

You seem to be confusing 'random' and 'ephemeral'.  

The whole reason I'm against adding this change is that it just makes the state file situation
that much worse.  Now instead of being able to control the labels either via code or configuration
management, we're going to trust humans to type things in at the command line. Essentially:
adding this feature *creates* the invalid label problem you are hoping to avoid!  

So in summary, again, I'm completely and totally against the ability to add labels from the
command line.  It's a big flaw in an implementation full of flaws.

> Create yarn cluster CLI to enable list node labels collection
> -------------------------------------------------------------
>
>                 Key: YARN-2786
>                 URL: https://issues.apache.org/jira/browse/YARN-2786
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: api, client, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-2786-20141031-1.patch, YARN-2786-20141031-2.patch, YARN-2786-20141102-2.patch,
YARN-2786-20141102-3.patch, YARN-2786-20141103-1-full.patch, YARN-2786-20141103-1-without-yarn.cmd.patch,
YARN-2786-20141104-1-full.patch, YARN-2786-20141104-1-without-yarn.cmd.patch, YARN-2786-20141104-2-full.patch,
YARN-2786-20141104-2-without-yarn.cmd.patch
>
>
> With YARN-2778, we can list node labels on existing RM nodes. But it is not enough, we
should be able to: 
> 1) list node labels collection
> The command should start with "yarn cluster ...", in the future, we can add more functionality
to the "yarnClusterCLI"



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message