hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8002) Support NOT_SELF and ALL namespace types for allocation tag
Date Sat, 10 Mar 2018 10:52:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394128#comment-16394128

Weiwei Yang commented on YARN-8002:

Hi [~leftnoteasy]

Thanks for the comments. Since #4 is going to result a major change, let me reply first
{quote}About existing aggregation approach
The approach in this patch adds the flexibility to let it easy to get cardinality on +a set
of application IDs+ regardless what namespace it is. The extra computation will only happen
when we need to compute PC against app-tag (or other namespace that evals to a set of app
IDs), not every time. And with the extra cost you mentioned (which I don't think it is very
bad). What you suggested saves this cost, but it will be hard to support +not-self, app-ids(a
set of IDs) or possibly app-regex+ etc. I'd like to hold on re-implementing this until every
one agrees we are not going to support those.

Regarding rest of comments
{quote}So how about calling it like "AllocationTagsFilter" or "AllocationTagsQuery"?
Hmm, if you look at the class, it contains 2 fields, one is {{AllocationTagNamespace}} and
the other is a set of allocation tags. This is naturally representing multiple allocation
tags within one same namespace. If you think this name is confusing, how about {{AllocationTagsWithNamespace}}?
But I prefer the current name.
{quote}About ALL evaluator:
Agree. Even in current patch, it checks if the namespace is ALL and directly retrieve from
global mapping if it is. So eval on ALL is not necessary. I can simply replace that with a
no OPT.
{quote}I'm not sure if NOT_SELF is necessary, unless it is must-to-have, I don't suggest to
add it.
I am OK with removing it, I am holding it for now in case [~asuresh] might give his opinion.
{quote}we should keep as less public interfaces as possible for ATM
Agree. Even I was planing to remove them, I did not include that in this patch was trying
to minimize the changes for easier review. I will address that in next patch.


> Support NOT_SELF and ALL namespace types for allocation tag
> -----------------------------------------------------------
>                 Key: YARN-8002
>                 URL: https://issues.apache.org/jira/browse/YARN-8002
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Major
>         Attachments: YARN-8002.001.patch, YARN-8002.002.patch
> This is a continua task after YARN-7972, YARN-7972 adds support to specify tags with
namespace SELF and APP_ID, like following
>  * self/<tag>
>  * app-id/<appid>/<tag>
> this task is to track the work to support 2 of remaining namespace types *NOT_SELF* &
*ALL* (we'll support app-label later),
>  * not-self/<tag>
>  * all/<tag>
> this will require a bit refactoring in {{AllocationTagsManager}} as it needs to do some
proper aggregation on tags for multiple apps.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message