hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3141) Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp
Date Thu, 15 Sep 2016 21:19:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494547#comment-15494547

Wangda Tan commented on YARN-3141:

Thanks [~templedf],

However I think for most of the comments, we should keep them as-is, a volatile varible doesn't
mean we don't need *extra lock to protect consistency between variables*. 
For a simplest example,
volatile boolean a;
volatile int b;

void update_b(b') {
	if (a) {
		b = b'

void update_a(a') {
	a = a'

boolean get_a() {
	return a; 

boolean get_b() {
	return b;

If two separate thread, thread #1 calls update_b first, and thread #2 calls update_a, it is
possible that update_a returns before update_b returns. And if we read the two variables,
data inconsistency happens.
So I would rather be more conservative: if a method read some fields and write some fields,
the safest way is add a single write lock to protect all them. Same to a method read multiple
fields, we should have a single readlock for them.

Most of the comments fall into the category, we could not shorten the critical sections of

What I have addressed in this patch:

bq. SchedulerApplicationAttempt.getLiveContainersMap() should be default visibility and @VisibleForTesting
bq. In FSAppAttempt.getAllowedLocalityLevel(), FSAppAttempt.getAllowedLocalityLevelByTime(),
FSAppAttempt.setReservation(), and FSAppAttempt.clearReservation() the write lock acquisition
can be delayed until after the arg validation
bq. There's an unused import in FaCiSchedulerApp

> Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp
> --------------------------------------------------------------------------
>                 Key: YARN-3141
>                 URL: https://issues.apache.org/jira/browse/YARN-3141
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-3141.1.patch, YARN-3141.2.patch, YARN-3141.3.patch
> Enhance locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp, as mentioned
in YARN-3091, a possible solution is using read/write lock. Other fine-graind locks for specific
purposes / bugs should be addressed in separated tickets.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message