hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zian Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8016) Refine PlacementRule interface and add a app-name queue mapping rule as an example
Date Mon, 19 Mar 2018 16:43:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405094#comment-16405094
] 

Zian Chen commented on YARN-8016:
---------------------------------

Just went through the failed cases and warnings about the second patch,

1. hadoop-yarn-api-warnings.html: org.apache.hadoop.yarn.api.records.Resource.getResources()
may expose internal representation by returning Resource.resources. {color:#FF0000}Have no
idea why the second patch affects this part.{color}

2. hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html: Redundant nullcheck of rule,
which is known to be non-null in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updatePlacementRules():
{color:#FF0000}will fix this issue in next patch{color}

3. For the failed UT cases, nearly all the cases were failed due to throw exception when
we call UserGroupMappingPlacementRule#initialize and didn't get any newMappings. Looks like
we can not force throwing the exception in UserGroupMappingPlacementRule#initialize to ensure
user to add UserGroupMappingPlacementRule into List<PlacementRule> placementRules data
structure if absent. What we can do here is add UserGroupMappingPlacementRule into placementRuleStrs
and do the switch case check, but if we do no have corresponding placement rule mapping values
setting for parmeter yarn.scheduler.capacity.queue-mappings, that should be totally fine.
I suggest we remove the throw expcetion inside UserGroupMappingPlacementRule#initialize and
return null if newMappings.size == 0 as we did it previously.

4. asflicense The patch generated 7 ASF License warnings: {color:#FF0000}have no idea how
to fix this issue.{color}

Could you share your opinions on these issues, [~leftnoteasy] ? Thank you!

> Refine PlacementRule interface and add a app-name queue mapping rule as an example
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-8016
>                 URL: https://issues.apache.org/jira/browse/YARN-8016
>             Project: Hadoop YARN
>          Issue Type: Task
>            Reporter: Zian Chen
>            Assignee: Zian Chen
>            Priority: Major
>         Attachments: YARN-8016.001.patch, YARN-8016.002.patch
>
>
> After YARN-3635/YARN-6689, PlacementRule becomes a common interface which can be used
by scheduler and can be dynamically updated by scheduler according to configs. There're some
other works. 
> - There's no way to initialize PlacementRule.
> - No example of PlacementRule except the user-group mapping one.
> This JIRA is targeted to refine PlacementRule interfaces and add another PlacementRule
example.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message