hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Deleted] (YARN-7854) Attach prefixes to different type of node attributes
Date Fri, 09 Feb 2018 01:51:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-7854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Wangda Tan updated YARN-7854:
    Comment: was deleted

(was: Hi [~LambertYe]

Thanks for the investigation. Following was from an earlier discussion

bq. Prefix will help in segregating the attributes provided by different sources and is a
DNS name, like nm.yarn.io/docker-supported,nm.yarn.io/container-executor-type, *.yarn.io is
limited to be defined by system instead of admin using centralized API.

So we need to set prefix name with DNS format.

bq. In RM merge from these sources, no need to handle the conflict cases.

This might be still necessary. [~Naganarasimha] raised his concern this morning about, for
example, nm1 reports an attribute A:STRING:V1, nm2 reports an attribute A:FLOAT:V2, is this
allowed? He suggested we need some cross-validation in RM on types. [~Naganarasimha], could
you please comment your idea?

bq. In placement constraints for node attributes should specify prefix to indicate the sources/namespaces,
or use yarn_nm/ prefix by default.

I agree. We need to define such behaviors. A rough thought is that in first phase, we can
ignore namespace check, that means an user is able to see all attributes (this is related
to constraints so we can visit this later).  User needs to specify the prefix while using
it (so it is able to get a match), otherwise a default prefix is added which belongs to this
user's namespace (I don't think we should use yarn_nm as default prefix for this case).


> Attach prefixes to different type of node attributes
> ----------------------------------------------------
>                 Key: YARN-7854
>                 URL: https://issues.apache.org/jira/browse/YARN-7854
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: RM
>            Reporter: Weiwei Yang
>            Assignee: LiangYe
>            Priority: Major
> There are multiple types of node attributes depending on which source it comes from,
>  # Centralized: attributes set by users (admin or normal users)
>  # Distributed: attributes collected by a certain attribute provider on each NM
>  # System: some built-in attributes in yarn, set by yarn internal components, e.g scheduler
> To better manage these attributes, we introduce the prefix (namespace) concept to the
an attribute. This Jira is opened to figure out how to attach prefixes (automatically/implicitly
or explicitly) to different type of attributes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message