hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8013) Support APP-TAG namespace for allocation tags
Date Sat, 31 Mar 2018 01:27:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16421128#comment-16421128

Weiwei Yang commented on YARN-8013:

Hi [~kkaranasos]

Thanks for providing your feedback.
{quote}AllocationTagNamespace is slightly misleading
That's the best name I came up with so far, I was hoping the java doc explains the bit (likely
not :( )
Class to describe the namespace of an allocation tag. Each namespace can be evaluated against
a set of applications.After evaluation, the namespace should have an implicit set of applications
which defines its scope.
Any suggested name? The reason I create this class because a tags' scope is lazy-binding,
on the other word, we cannot get the scope during the parsing phase of a PC expression. That's
why we need a eval phase, that can help to separate different "eval" logic in the different
implementation of the namespace class.
{quote}Why not have a method inside the tags manager that takes an nsScope and returns only
the target applications for that nsScope
The best I can do is to move the namespace eval step into allocation tags manager, that will
help to avoid exposing \{{AllocationTagsManager#getApplicationIdToTags}}. The reason I pass
in \{{AllocationTags}} instead of just target application IDs to #getCardinality methods was
because when namespace is ALL, it needs to use the global mapping, which means it needs to
know the namespace type of the tags.
{quote}It seems that this will simplify the code of {{TargetApplications}} too.
I can even remove this class by letting namespace to be eval'd by two args 1) current appId
2) all appIds. But I thought this is better encapsulated could be easier for reading. What
do you think?
{quote} perAppNodeMappings.keySet().contains(){quote}


> Support APP-TAG namespace for allocation tags
> ---------------------------------------------
>                 Key: YARN-8013
>                 URL: https://issues.apache.org/jira/browse/YARN-8013
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>            Priority: Major
>         Attachments: YARN-8013.001.patch, YARN-8013.002.patch, YARN-8013.003.patch, YARN-8013.004.patch
> YARN-1461 adds *Application Tag* concept to Yarn applications, user is able to annotate
application with multiple tags to classify apps. We can leverage this to represent a namespace
for a certain group of apps. So instead of calling *app-label*, propose to call it *app-tag*.
> A typical use case is,
> There are a lot of TF jobs running on Yarn, and some of them are consuming resources
heavily. So we want to limit number of PS on each node for such BIG players but ignore those
SMALL ones. To achieve this, we can do following steps:
>  # Add application tag "big-tf" to these big TF jobs
>  # For each PS request, we add "ps" source tag and map it to constraint "{color:#d04437}notin,
node, tensorflow/ps{color}" or "{color:#d04437}cardinality, node, tensorflow/ps{color}{color:#d04437},
0, 2{color}" for finer grained controls.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message