hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil Govindan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8948) PlacementRule interface should be for all YarnSchedulers
Date Fri, 02 Nov 2018 10:26:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-8948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672883#comment-16672883
] 

Sunil Govindan commented on YARN-8948:
--------------------------------------

[~bibinchundatt]

Few questions:
 # cud u pls explain how this is breaking wire compatibility b/w old hadoop version to new
one?
 # These jiras are already in hadoop-3.1 and other released? Hence is this already broken
by these version?
 # any workarounds if u upgrade to hadoop-3.2 and then use old clients/apps by which we ll
not have any breakage?

If this is not breaking any compatibility, i would like to degrade severity to Minor so that
3.2 release can go on.

cc [~leftnoteasy] [~suma.shivaprasad]

> PlacementRule interface should be for all YarnSchedulers
> --------------------------------------------------------
>
>                 Key: YARN-8948
>                 URL: https://issues.apache.org/jira/browse/YARN-8948
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>            Priority: Critical
>         Attachments: YARN-8948.001.patch, YARN-8948.002.patch, YARN-8948.003.patch
>
>
> *Issue 1:*
> YARN-3635 intention was to add PlacementRule interface common for all YarnSchedules.
> {code}
> 33	  public abstract boolean initialize(
> 34	      CapacitySchedulerContext schedulerContext) throws IOException;
> {code}
> PlacementRule initialization is done using CapacitySchedulerContext binding to CapacityScheduler
> *Issue 2:*
> {{yarn.scheduler.queue-placement-rules}} doesn't work as expected in Capacity Scheduler
> {quote}
> * **Queue Mapping Interface based on Default or User Defined Placement Rules** - This
feature allows users to map a job to a specific queue based on some default placement rule.
For instance based on user & group, or application name. User can also define their own
placement rule.
> {quote}
> As per current UserGroupMapping is always added in placementRule. {{CapacityScheduler#updatePlacementRules}}
> {code}
>     // Initialize placement rules
>     Collection<String> placementRuleStrs = conf.getStringCollection(
>         YarnConfiguration.QUEUE_PLACEMENT_RULES);
>     List<PlacementRule> placementRules = new ArrayList<>();
> ...
>     // add UserGroupMappingPlacementRule if absent
>     distingushRuleSet.add(YarnConfiguration.USER_GROUP_PLACEMENT_RULE);
> {code}
> PlacementRule configuration order is not maintained 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message