hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szilard Nemeth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-9318) Resources#multiplyAndRoundUp does not consider Resource Types
Date Sun, 24 Feb 2019 10:38:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16776217#comment-16776217

Szilard Nemeth commented on YARN-9318:

Hi [~sunilg]!

Sure, good idea as these are indeed similar and have code duplication.
I introduced two methods: 
- applyFunctionOnValues (with one Resource): Runs through all the countable resources and
updates the individual values with the result of a value function, then saves the result into
the same Resource object with the updated resource value.
- applyFunctionOnValues (with two Resources): Does almost the same as above function but takes
two Resources (lhs, rhs) and transforms the value and saves the result into lhs.
With these, I could achieve less code duplication.
Please note that these 2 new methods could be used for other methods like componentwiseMin,
componentwiseMax, addTo and subtractFrom. These methods also suffer from code duplication
with a similar reason. If you agree, I would open a follow-up jira to reduce the code duplication
for those methods, after this one gets merged.

I touched the following methods with the latest patch:
- multiplyTo: Now invokes applyFunctionOnValues with a simple mulpiplier lambda.
- multiply: Simply calls multiplyTo.
- multiplyAndRoundDown: I just dispatched the call to multiplyTo. Maybe this method could
be deleted in the future as it basically does the same as multiplyTo.
- multiplyAndRoundUp: Now invokes applyFunctionOnValues with a simple mulpiplier lambda that
also performs Math.ceil.

> Resources#multiplyAndRoundUp does not consider Resource Types
> -------------------------------------------------------------
>                 Key: YARN-9318
>                 URL: https://issues.apache.org/jira/browse/YARN-9318
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Szilard Nemeth
>            Assignee: Szilard Nemeth
>            Priority: Major
>         Attachments: YARN-9318.001.patch
> org.apache.hadoop.yarn.util.resource.Resources#multiplyAndRoundUp only deals with memory
and vcores while computing the rounded value. It should also consider custom Resource Types
as well.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message