hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szilard Nemeth (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-7528) Resource types that use units need to be defined at RM level and NM level or when using small units you will overflow max_allocation calculation
Date Wed, 17 Jan 2018 09:20:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-7528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328498#comment-16328498
] 

Szilard Nemeth edited comment on YARN-7528 at 1/17/18 9:19 AM:
---------------------------------------------------------------

[~gsohn] No luck. 
I tried with this resource-types.xml: 


{code:java}
<configuration>
<property>
   <name>yarn.resource-types</name>
   <value>gpu,fpga</value>
 </property>
 
 <property>
   <name>yarn.resource-types.gpu.units</name>
   <value>m</value>
 </property>
</configuration>
{code}

I ran the job with these parameters: "pi -Dmapreduce.framework.name=yarn -Dmapreduce.map.resource.gpu=5000
10 100"
The exception does not come up and the job does not hang.

If you have time, you can check what scenarios I have tried so far in my previous comment.
Do you agree to close this as non-reproducible?


was (Author: snemeth):
No luck. 
I tried with this resource-types.xml: 


{code:java}
<configuration>
<property>
   <name>yarn.resource-types</name>
   <value>gpu,fpga</value>
 </property>
 
 <property>
   <name>yarn.resource-types.gpu.units</name>
   <value>m</value>
 </property>
</configuration>
{code}

I ran the job with these parameters: "pi -Dmapreduce.framework.name=yarn -Dmapreduce.map.resource.gpu=5000
10 100"
The exception does not come up and the job does not hang.


> Resource types that use units need to be defined at RM level and NM level or when using
small units you will overflow max_allocation calculation
> ------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-7528
>                 URL: https://issues.apache.org/jira/browse/YARN-7528
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: documentation, resourcemanager
>    Affects Versions: 3.0.0
>            Reporter: Grant Sohn
>            Assignee: Szilard Nemeth
>            Priority: Major
>
> When the unit is not defined in the RM, the LONG_MAX default will overflow in the conversion
step.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message