hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5375) invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures
Date Wed, 24 Aug 2016 18:27:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-5375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15435429#comment-15435429

Varun Saxena commented on YARN-5375:

There are many events where draining of both rm dispatcher and state store dispatcher is not
required and as Rohith pointed out in some places, we may need to drain RM Dispatcher after
state store dispatcher drain as well. There could be different sequences as well. Should we
create 2 drain methods and call each one as per need ?

Also many of the waitForState can be eliminated by calling a sequence of drains. For different
kind of events, we may have to decide what do we do.
Maybe for some regularly checked states, we can add some specific methods in MockRM in addition
to generic waitForState methods with customary draining of RM dispatcher and state store dispatcher
and again RM dispatcher. This as you said can reduce test times.
Thoughts ?

For instance, after Node HB we typically check if an attempt has transferred to ALLOCATED
state in many of the tests. Here, first rm dispatcher must be drained to process node update
to be sent to Scheduler. As with the changes in patch, scheduler events will be processed
immediately, it will lead to App Attempt event for Container being allocated (from RM Container).
This would again require draining of RM dispatcher queue which would lead to a state store
event and from State store we will again send a ATTEMPT_SAVED event back to RM App attempt,
which would again require draining of RM dispatcher queue. And only then will attempt state
change to ALLOCATED.

> invoke MockRM#drainEvents implicitly in MockRM methods to reduce test failures
> ------------------------------------------------------------------------------
>                 Key: YARN-5375
>                 URL: https://issues.apache.org/jira/browse/YARN-5375
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: sandflee
>            Assignee: sandflee
>         Attachments: YARN-5375.01.patch, YARN-5375.03.patch, YARN-5375.04.patch, YARN-5375.05.patch,
> seen many test failures related to RMApp/RMAppattempt comes to some state but some event
are not processed in rm event queue or scheduler event queue, cause test failure, seems we
could implicitly invokes drainEvents(should also drain sheduler event) in some mockRM method
like waitForState

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message