Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 29 Aug 2016 01:25:20 +0000 (UTC)
From: "sandflee (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12989315.1468460228000.432083.1472433920505@Atlassian.JIRA>
In-Reply-To: <JIRA.12989315.1468460228000@Atlassian.JIRA>
References: <JIRA.12989315.1468460228000@Atlassian.JIRA> <JIRA.12989315.1468460228204@arcas>
Subject: [jira] [Comment Edited] (YARN-5375) invoke MockRM#drainEvents
 implicitly in MockRM methods to reduce test failures
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 29 Aug 2016 01:25:22 -0000


    [ https://issues.apache.org/jira/browse/YARN-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15444465#comment-15444465 ] 

sandflee edited comment on YARN-5375 at 8/29/16 1:24 AM:
---------------------------------------------------------

thanks [~varun_saxena], [~sunilg], [~rohithsharma] for your comment and suggest !  , update two patches. 
1, add MockRMMemoryStateStore in MockRM
"drain patch" adds a DrainDispatcher and will call rm-dispatcher.await, statestore-dispatcher.await rm-dispatcher.await when drainEvents. this works for almost all of cases
"sync patch" makes stateStore Event processed in a sync way. so drainEvents will drain all events, this will drain some unnessesary events, but seems a more general way.
2, accessing DrainDispatcher#drained should be protected by mutex, or there will be a race condition.


was (Author: sandflee):
thanks [~varun_saxena], [~sunilg], [~rohithsharma] for your comment and suggest !  
1, add MockRMMemoryStateStore,
"drain patch" adds a DrainDispatcher and will call rm-dispatcher.await, statestore-dispatcher.await rm-dispatcher.await when drainEvents. this works for almost all of cases
"sync patch" makes stateStore Event processed in a sync way. so drainEvents will drain all events, this will drain some unnessesary events, but seems a more general way.
2, accessing DrainDispatcher#drained should be protected by mutex, or there will be a race condition.


> invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures
> ------------------------------------------------------------------------------
>
>                 Key: YARN-5375
>                 URL: https://issues.apache.org/jira/browse/YARN-5375
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: sandflee
>            Assignee: sandflee
>         Attachments: YARN-5375.01.patch, YARN-5375.03.patch, YARN-5375.04.patch, YARN-5375.05.patch, YARN-5375.06.patch, YARN-5375.07-drain-statestore.patch, YARN-5375.07-sync-statestore.patch
>
>
> seen many test failures related to RMApp/RMAppattempt comes to some state but some event are not processed in rm event queue or scheduler event queue, cause test failure, seems we could implicitly invokes drainEvents(should also drain sheduler event) in some mockRM method like waitForState


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org