Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 19 May 2016 11:44:13 +0000 (UTC)
From: "Rohith Sharma K S (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12850485.1438309492000.235098.1463658253666@Atlassian.JIRA>
In-Reply-To: <JIRA.12850485.1438309492000@Atlassian.JIRA>
References: <JIRA.12850485.1438309492000@Atlassian.JIRA> <JIRA.12850485.1438309492917@arcas>
Subject: [jira] [Updated] (YARN-4002) make
 ResourceTrackerService.nodeHeartbeat more concurrent
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 19 May 2016 11:44:15 -0000


     [ https://issues.apache.org/jira/browse/YARN-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rohith Sharma K S updated YARN-4002:
------------------------------------
    Attachment: YARN-4002-rwlock-v6.patch

Updating the patch with below changes from previous patch
# fixed findbugs and checkstyle warning
# for the setter methods, removed writelock

> make ResourceTrackerService.nodeHeartbeat more concurrent
> ---------------------------------------------------------
>
>                 Key: YARN-4002
>                 URL: https://issues.apache.org/jira/browse/YARN-4002
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Hong Zhiguo
>            Assignee: Hong Zhiguo
>            Priority: Critical
>         Attachments: 0001-YARN-4002.patch, YARN-4002-lockless-read.patch, YARN-4002-rwlock-v2.patch, YARN-4002-rwlock-v2.patch, YARN-4002-rwlock-v3-rebase.patch, YARN-4002-rwlock-v3.patch, YARN-4002-rwlock-v4.patch, YARN-4002-rwlock-v5.patch, YARN-4002-rwlock-v6.patch, YARN-4002-rwlock.patch, YARN-4002-v0.patch
>
>
> We have multiple RPC threads to handle NodeHeartbeatRequest from NMs. By design the method ResourceTrackerService.nodeHeartbeat should be concurrent enough to scale for large clusters.
> But we have a "BIG" lock in NodesListManager.isValidNode which I think it's unnecessary.
> First, the fields "includes" and "excludes" of HostsFileReader are only updated on "refresh nodes".  All RPC threads handling node heartbeats are only readers.  So RWLock could be used to  alow concurrent access by RPC threads.
> Second, since he fields "includes" and "excludes" of HostsFileReader are always updated by "reference assignment", which is atomic in Java, the reader side lock could just be skipped.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org