hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7307) Revisit resource-types.xml loading behaviors
Date Mon, 16 Oct 2017 18:26:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16206347#comment-16206347
] 

Wangda Tan edited comment on YARN-7307 at 10/16/17 6:25 PM:
------------------------------------------------------------

Thanks [~sunilg] for updating the patch, 

Few comments:

1) Could we add a new static method to ResourceUtils to initialize from ResourceTypeInfo directly,
and move logics below to the method:
{code}
	225	    List<ResourceTypeInfo> resourceTypes = getResourceTypeInfo();
226	    Map<String, ResourceInformation> resourceInformationMap = new HashMap<>();
227	
228	    for (ResourceTypeInfo resourceType : resourceTypes) {
229	      resourceInformationMap.put(resourceType.getName(),
230	          ResourceInformation.newInstance(resourceType.getName(),
231	              resourceType.getDefaultUnit(), resourceType.getResourceType()));
232	    }
{code}

2) This patch updated YarnClientImpl service start to set resource types directly. I'm not
sure if we should move this to a separate method so client can opt-out initialize resource
types. I can see the biggest advantage of your approach is client can get resource types updated
without changing any code if YarnClient initialized and started before any resource object
created. However since applications need to update their code in order to use this feature
anyway, I'm not sure if it is the best option to add this call inside serviceStart. 

Another (safer) option is to add a boolean flag to getResourceTypeInfo to indicate that if
client want to update ResourceUtils with the method call or not. 

The same comment applies to AM.

3) Do you want to add changes to AMRMClient for now or move it to a separate ticket?

(I prefer to move it to a separate ticket.)


was (Author: leftnoteasy):
Thanks [~sunilg] for updating the patch, 

Few comments:

1) Could we add a new static method to ResourceUtils to initialize from ResourceTypeInfo directly,
and move logics below to the method:
{code}
	225	    List<ResourceTypeInfo> resourceTypes = getResourceTypeInfo();
226	    Map<String, ResourceInformation> resourceInformationMap = new HashMap<>();
227	
228	    for (ResourceTypeInfo resourceType : resourceTypes) {
229	      resourceInformationMap.put(resourceType.getName(),
230	          ResourceInformation.newInstance(resourceType.getName(),
231	              resourceType.getDefaultUnit(), resourceType.getResourceType()));
232	    }
{code}

2) This patch updated YarnClientImpl service start to set resource types directly. I'm not
sure if we should move this to a separate method so client can opt-out initialize resource
types. I can see the biggest advantage of your approach is client can get resource types updated
without changing any code if YarnClient initialized and started before any resource object
created. However since applications need to update their code in order to use this feature
anyway, I'm not sure if it is the best option to add this call inside serviceStart. 

Another (safer) option is to add a boolean flag to getResourceTypeInfo to indicate that if
client want to update ResourceUtils with the method call or not. 

The same comment applies to AM.

3) Do you want to add changes to AMRMClient for now or move it to a separate ticket?

> Revisit resource-types.xml loading behaviors
> --------------------------------------------
>
>                 Key: YARN-7307
>                 URL: https://issues.apache.org/jira/browse/YARN-7307
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Sunil G
>            Priority: Blocker
>         Attachments: YARN-7307.001.patch, YARN-7307.002.patch
>
>
> Existing feature requires every client has a resource-types.xml in order to use multiple
resource types, should we allow client/AM update supported resource types via Yarn APIs?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message