hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7410) Cleanup FixedValueResource to avoid dependency to ResourceUtils
Date Mon, 30 Oct 2017 19:04:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16225557#comment-16225557

Wangda Tan commented on YARN-7410:

Thanks [~sunilg] for trying this patch. 

I just updated description. The major reason to reinit resource map is, FixedValueResource
cannot be updated when any resource types refresh by using approach introduced by YARN-7307
and do operations like {{Resources.compare(resource_x, Resources.none())}} will throw exceptions.
Please let me know if you have any suggestions/concerns of this approach. 

I will update patch to address #1/#2. 

And also, discussed with [~sunilg] last week regarding to the approach. This is a short term
solution. Even after this patch, we still have issues of FixedValueResource: 
1) Returned array is still mutable which means returned array can be modified by callers and
cause hard-to-debug issues. 
2) We still calculate fair share, etc. while comparing the FixedValuedResources to other resource
We need to consider solutions of these issues in the longer term.

> Cleanup FixedValueResource to avoid dependency to ResourceUtils
> ---------------------------------------------------------------
>                 Key: YARN-7410
>                 URL: https://issues.apache.org/jira/browse/YARN-7410
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 3.1.0
>            Reporter: Sunil G
>            Assignee: Wangda Tan
>         Attachments: YARN-7410.001.patch, YARN-7410.002.patch
> After YARN-7307, Client/AM don't need to keep a up-to-dated resource-type.xml in the
classpath. Instead, they can use YarnClient/ApplicationMasterProtocol APIs to get the resource
types from RM and refresh local types.
> One biggest issue of this approach is FixedValueResource: Since we initialize FixedValueResource
in static block, and they won't be updated if resource types refreshed.
> So we need to properly update FixedValueResource to make it can get up-to-date results

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message