hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bibin A Chundatt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8948) PlacementRule interface should be for all YarnSchedulers
Date Fri, 02 Nov 2018 10:52:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16672922#comment-16672922
] 

Bibin A Chundatt commented on YARN-8948:
----------------------------------------

[~sunilg]

{code}
cud u pls explain how this is breaking wire compatibility
{code}
Not breaking wire compatability since nothing is transmitted between process.

In jira i mentioned the *interface is supposed be for all schedulers.* shouldnt be for capacityscheduler
alone. Else next version  might have to change again?? 

{code}
If this is not breaking any compatibility, i would like to degrade severity to Minor so that
3.2 release can go on.
{code}
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html

Since we havnt marked {{PlacementRule}} as public interface level compatability might not
be requried. 

YARN-8016 made the placementRule refined and made it usable.

Another issue seems currently UserGroup*Rule is added always and the order is not maintained
for {{distingushRuleSet}} custom configuration will not work as expected. Consider as functional
issue.

Lets take a call based on response from [~leftnoteasy] and [~suma.shivaprasad] too.

> PlacementRule interface should be for all YarnSchedulers
> --------------------------------------------------------
>
>                 Key: YARN-8948
>                 URL: https://issues.apache.org/jira/browse/YARN-8948
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>            Priority: Critical
>         Attachments: YARN-8948.001.patch, YARN-8948.002.patch, YARN-8948.003.patch
>
>
> *Issue 1:*
> YARN-3635 intention was to add PlacementRule interface common for all YarnSchedules.
> {code}
> 33	  public abstract boolean initialize(
> 34	      CapacitySchedulerContext schedulerContext) throws IOException;
> {code}
> PlacementRule initialization is done using CapacitySchedulerContext binding to CapacityScheduler
> *Issue 2:*
> {{yarn.scheduler.queue-placement-rules}} doesn't work as expected in Capacity Scheduler
> {quote}
> * **Queue Mapping Interface based on Default or User Defined Placement Rules** - This
feature allows users to map a job to a specific queue based on some default placement rule.
For instance based on user & group, or application name. User can also define their own
placement rule.
> {quote}
> As per current UserGroupMapping is always added in placementRule. {{CapacityScheduler#updatePlacementRules}}
> {code}
>     // Initialize placement rules
>     Collection<String> placementRuleStrs = conf.getStringCollection(
>         YarnConfiguration.QUEUE_PLACEMENT_RULES);
>     List<PlacementRule> placementRules = new ArrayList<>();
> ...
>     // add UserGroupMappingPlacementRule if absent
>     distingushRuleSet.add(YarnConfiguration.USER_GROUP_PLACEMENT_RULE);
> {code}
> PlacementRule configuration order is not maintained 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message