hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6788) Improve performance of resource profile branch
Date Mon, 31 Jul 2017 20:12:00 GMT

     [ https://issues.apache.org/jira/browse/YARN-6788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Sunil G updated YARN-6788:
    Attachment: YARN-6788-YARN-3926.018.patch

Thanks [~templedf] for comments. Updating new patch fixing same.

bq. What I was trying to say is that getResourceTypesArray() starts with a call to getResourceTypes() to
make sure the array is initialized before returning it. Why doesn't getResourceNamesArray() also
do that?
In {{DominantResourceCalculator()}}, we already invoke {{getResourceTypes}}. So i think its

bq.Sorry to be slow, but I still don't see it. What I mean is that knownResourceNamesArray
is initialized like this
Yes. You are correct,

bq.The risk is that if thread A is just exiting the synchronized block, and thread B is hitting
the first check, thread B may see that lock is initialized, but knownResourceNamesArray may
not have been flushed yet.
I think i understood the point. But as mentioned in above statement, thread A is just exiting
the synchronized block, then resourceTypesArray/NamesArray must be initialized since its inside
synchronized block. Yes, ideally we can optimize here and change to a volatile boolean + array
object. If its fine, i would like to take that in another ticket so that i can run some thread
analyzer tools as well. Thoughts?

> Improve performance of resource profile branch
> ----------------------------------------------
>                 Key: YARN-6788
>                 URL: https://issues.apache.org/jira/browse/YARN-6788
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Sunil G
>            Assignee: Sunil G
>            Priority: Blocker
>         Attachments: YARN-6788-YARN-3926.001.patch, YARN-6788-YARN-3926.002.patch, YARN-6788-YARN-3926.003.patch,
YARN-6788-YARN-3926.004.patch, YARN-6788-YARN-3926.005.patch, YARN-6788-YARN-3926.006.patch,
YARN-6788-YARN-3926.007.patch, YARN-6788-YARN-3926.008.patch, YARN-6788-YARN-3926.009.patch,
YARN-6788-YARN-3926.010.patch, YARN-6788-YARN-3926.011.patch, YARN-6788-YARN-3926.012.patch,
YARN-6788-YARN-3926.013.patch, YARN-6788-YARN-3926.014.patch, YARN-6788-YARN-3926.015.patch,
YARN-6788-YARN-3926.016.patch, YARN-6788-YARN-3926.017.patch, YARN-6788-YARN-3926.018.patch
> Currently we could see a 15% performance delta with this branch. 
> Few performance improvements to improve the same.
> Also this patch will handle [comments|https://issues.apache.org/jira/browse/YARN-6761?focusedCommentId=16075418&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16075418]
from [~leftnoteasy].

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message