hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6680) Avoid locking overhead for NO_LABEL lookups
Date Fri, 02 Jun 2017 20:10:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-6680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16035337#comment-16035337
] 

Naganarasimha G R commented on YARN-6680:
-----------------------------------------

Thanks for the clarification [~daryn], my query was majorly related to see the CPU profiler
output based on which these modifications are done as earlier you had just mentioned 3x performance
decrease.
And good to hear that its already been reviewed by [~jlowe] and [~nroberts] , But just for
my clarification could you help me in understanding following points:
# IIUC , the RW locks of CommonNodeLabelsManager is same as in 2.7, so wonder why the performance
was impacted if you have migrated from 2.7.x to 2.8. 
# ??Eventually these maps should be concurrent maps which uses no lock for read ops, and the
memory read barriers are cheap?? {{labelCollections}} and {{nodeCollections}} are already
concurrentmaps. so you meant RW locks are not required?
# Assume we solve the issue for NO_LABEL but still the flow is same for other labels. So issue
will still be present with flows for other labels?
# did not quite get your consistent and Inconsistent example though agree that the resource
object got during read gets modified using the write lock as the same resource object is being
returned instead of a clone. 

cc [~wangda]

> Avoid locking overhead for NO_LABEL lookups
> -------------------------------------------
>
>                 Key: YARN-6680
>                 URL: https://issues.apache.org/jira/browse/YARN-6680
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.8.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>         Attachments: YARN-6680.patch
>
>
> Labels are managed via a hash that is protected with a read lock.  The lock acquire and
release are each just as expensive as the hash lookup itself - resulting in a 3X slowdown.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message