hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3141) Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp
Date Thu, 15 Sep 2016 21:19:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-3141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15494547#comment-15494547
] 

Wangda Tan commented on YARN-3141:
----------------------------------

Thanks [~templedf],

However I think for most of the comments, we should keep them as-is, a volatile varible doesn't
mean we don't need *extra lock to protect consistency between variables*. 
For a simplest example,
{code}
volatile boolean a;
volatile int b;

void update_b(b') {
	if (a) {
		b = b'
	}
}

void update_a(a') {
	a = a'
}

boolean get_a() {
	return a; 
}

boolean get_b() {
	return b;
}
{code} 

If two separate thread, thread #1 calls update_b first, and thread #2 calls update_a, it is
possible that update_a returns before update_b returns. And if we read the two variables,
data inconsistency happens.
So I would rather be more conservative: if a method read some fields and write some fields,
the safest way is add a single write lock to protect all them. Same to a method read multiple
fields, we should have a single readlock for them.

Most of the comments fall into the category, we could not shorten the critical sections of
them.

What I have addressed in this patch:

bq. SchedulerApplicationAttempt.getLiveContainersMap() should be default visibility and @VisibleForTesting
bq. In FSAppAttempt.getAllowedLocalityLevel(), FSAppAttempt.getAllowedLocalityLevelByTime(),
FSAppAttempt.setReservation(), and FSAppAttempt.clearReservation() the write lock acquisition
can be delayed until after the arg validation
bq. There's an unused import in FaCiSchedulerApp
bq. 

> Improve locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp
> --------------------------------------------------------------------------
>
>                 Key: YARN-3141
>                 URL: https://issues.apache.org/jira/browse/YARN-3141
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager, scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-3141.1.patch, YARN-3141.2.patch, YARN-3141.3.patch
>
>
> Enhance locks in SchedulerApplicationAttempt/FSAppAttempt/FiCaSchedulerApp, as mentioned
in YARN-3091, a possible solution is using read/write lock. Other fine-graind locks for specific
purposes / bugs should be addressed in separated tickets.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message