hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bibin A Chundatt (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-8948) PlacementRule interface should be for all YarnSchedulers
Date Fri, 02 Nov 2018 07:30:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-8948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Bibin A Chundatt updated YARN-8948:
-----------------------------------
    Description: 
*Issue 1:*

YARN-3635 intention was to add PlacementRule interface common for all YarnSchedules.
{code}
33	  public abstract boolean initialize(
34	      CapacitySchedulerContext schedulerContext) throws IOException;
{code}
PlacementRule initialization is done using CapacitySchedulerContext binding to CapacityScheduler

*Issue 2:*

{{yarn.scheduler.queue-placement-rules}} doesn't work as expected in Capacity Scheduler
{quote}
* **Queue Mapping Interface based on Default or User Defined Placement Rules** - This feature
allows users to map a job to a specific queue based on some default placement rule. For instance
based on user & group, or application name. User can also define their own placement rule.
{quote}
As per current UserGroupMapping is always added in placementRule. {{CapacityScheduler#updatePlacementRules}}
{code}
    // Initialize placement rules
    Collection<String> placementRuleStrs = conf.getStringCollection(
        YarnConfiguration.QUEUE_PLACEMENT_RULES);
    List<PlacementRule> placementRules = new ArrayList<>();
...

    // add UserGroupMappingPlacementRule if absent
    distingushRuleSet.add(YarnConfiguration.USER_GROUP_PLACEMENT_RULE);
{code}
PlacementRule configuration order is not maintained 

  was:
*Issue 1:*

YARN-3635 intention was to add PlacementRule interface common for all YarnSchedules.
{code}
33	  public abstract boolean initialize(
34	      CapacitySchedulerContext schedulerContext) throws IOException;
{code}
PlacementRule initialization is done using CapacitySchedulerContext binding to CapacityScheduler

*Issue 2:*

{{yarn.scheduler.queue-placement-rules}} doesn't work as expected in Capacity Scheduler
{quote}
* **Queue Mapping Interface based on Default or User Defined Placement Rules** - This feature
allows users to map a job to a specific queue based on some default placement rule. For instance
based on user & group, or application name. User can also define their own placement rule.
{quote}
As per current UserGroupMapping is always added in placementRule. {{CapacityScheduler#updatePlacementRules}}
{code}
    // Initialize placement rules
    Collection<String> placementRuleStrs = conf.getStringCollection(
        YarnConfiguration.QUEUE_PLACEMENT_RULES);
    List<PlacementRule> placementRules = new ArrayList<>();
...

    // add UserGroupMappingPlacementRule if absent
    distingushRuleSet.add(YarnConfiguration.USER_GROUP_PLACEMENT_RULE);
{code}


> PlacementRule interface should be for all YarnSchedulers
> --------------------------------------------------------
>
>                 Key: YARN-8948
>                 URL: https://issues.apache.org/jira/browse/YARN-8948
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Priority: Critical
>         Attachments: YARN-8948.001.patch, YARN-8948.002.patch
>
>
> *Issue 1:*
> YARN-3635 intention was to add PlacementRule interface common for all YarnSchedules.
> {code}
> 33	  public abstract boolean initialize(
> 34	      CapacitySchedulerContext schedulerContext) throws IOException;
> {code}
> PlacementRule initialization is done using CapacitySchedulerContext binding to CapacityScheduler
> *Issue 2:*
> {{yarn.scheduler.queue-placement-rules}} doesn't work as expected in Capacity Scheduler
> {quote}
> * **Queue Mapping Interface based on Default or User Defined Placement Rules** - This
feature allows users to map a job to a specific queue based on some default placement rule.
For instance based on user & group, or application name. User can also define their own
placement rule.
> {quote}
> As per current UserGroupMapping is always added in placementRule. {{CapacityScheduler#updatePlacementRules}}
> {code}
>     // Initialize placement rules
>     Collection<String> placementRuleStrs = conf.getStringCollection(
>         YarnConfiguration.QUEUE_PLACEMENT_RULES);
>     List<PlacementRule> placementRules = new ArrayList<>();
> ...
>     // add UserGroupMappingPlacementRule if absent
>     distingushRuleSet.add(YarnConfiguration.USER_GROUP_PLACEMENT_RULE);
> {code}
> PlacementRule configuration order is not maintained 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message