hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6789) new api to get all supported resources from RM
Date Thu, 10 Aug 2017 18:00:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16121998#comment-16121998
] 

Wangda Tan commented on YARN-6789:
----------------------------------

[~daniel@cloudera.com],  thanks for sharing your thoughts. Please see my response below:

bq. If I understand correctly, what you're proposing is making the units a global dictate.
The RM says that memory has the unit "Mi", therefore all NMs and clients must use that unit.

This is true, unit of values for different resource types are fixed. To be more clear, I propose
to remove the "unit" from ResourceInformation object. "Unit" for each resource types are unwritten
rules which each daemon/clients/am should know that. "Unit" can be set for ResourceTypeInformation
(by setting resource-types.xml), however it is more like documentation instead of affecting
internal logics.

bq. What happens if an admin changes the units for a resource type? 
This is not allowed in my proposal, I doubt that in what scenario we need to change global
units. For example, MB as unit for memory works so well today, I don't think it make sense
to upgrade it to GB/TB since value is a 64 bits integer.
This is majorly to avoid dealing with backward compatibility and inconsistency between daemons
and clients. As I stated above, what if an old client talk to RM asks 4096 memory without
specifying "MB" as unit? If we make it to be globally fixed and unwritten rule, we don't need
to worry about inconsistency any more. And all rolling upgrade problems are gone for the unit.


Thoughts?

> new api to get all supported resources from RM
> ----------------------------------------------
>
>                 Key: YARN-6789
>                 URL: https://issues.apache.org/jira/browse/YARN-6789
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-6789-YARN-3926.001.patch
>
>
> It will be better to provide an api to get all supported resource types from RM.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message