hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sandflee (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5375) invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures
Date Mon, 29 Aug 2016 01:25:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15444465#comment-15444465
] 

sandflee edited comment on YARN-5375 at 8/29/16 1:24 AM:
---------------------------------------------------------

thanks [~varun_saxena], [~sunilg], [~rohithsharma] for your comment and suggest !  , update
two patches. 
1, add MockRMMemoryStateStore in MockRM
"drain patch" adds a DrainDispatcher and will call rm-dispatcher.await, statestore-dispatcher.await
rm-dispatcher.await when drainEvents. this works for almost all of cases
"sync patch" makes stateStore Event processed in a sync way. so drainEvents will drain all
events, this will drain some unnessesary events, but seems a more general way.
2, accessing DrainDispatcher#drained should be protected by mutex, or there will be a race
condition.



was (Author: sandflee):
thanks [~varun_saxena], [~sunilg], [~rohithsharma] for your comment and suggest !  
1, add MockRMMemoryStateStore,
"drain patch" adds a DrainDispatcher and will call rm-dispatcher.await, statestore-dispatcher.await
rm-dispatcher.await when drainEvents. this works for almost all of cases
"sync patch" makes stateStore Event processed in a sync way. so drainEvents will drain all
events, this will drain some unnessesary events, but seems a more general way.
2, accessing DrainDispatcher#drained should be protected by mutex, or there will be a race
condition.


> invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures
> ------------------------------------------------------------------------------
>
>                 Key: YARN-5375
>                 URL: https://issues.apache.org/jira/browse/YARN-5375
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: sandflee
>            Assignee: sandflee
>         Attachments: YARN-5375.01.patch, YARN-5375.03.patch, YARN-5375.04.patch, YARN-5375.05.patch,
YARN-5375.06.patch, YARN-5375.07-drain-statestore.patch, YARN-5375.07-sync-statestore.patch
>
>
> seen many test failures related to RMApp/RMAppattempt comes to some state but some event
are not processed in rm event queue or scheduler event queue, cause test failure, seems we
could implicitly invokes drainEvents(should also drain sheduler event) in some mockRM method
like waitForState



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message