hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maysam Yabandeh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1069) Deterministic MiniYARNCluster
Date Sat, 17 Aug 2013 07:08:48 GMT

    [ https://issues.apache.org/jira/browse/YARN-1069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13742865#comment-13742865
] 

Maysam Yabandeh commented on YARN-1069:
---------------------------------------

Thanks for the pointer [~vinodkv]. The MockRM and MockNM work great when we are testing the
individual components at the RM side. There are cases, however, that we need an end-to-end
testing including both client code at AM and the server code at the RM. For example, the bug
in YARN-779 is due to the way the AMRMClientImpl translates a ContainerRequest to multiple
resource requests to be sent to the RM. In such cases, we need the MiniYarnCluster, as the
similar tests in TestAMRMClient are using it. But, we face the non-determinism problem that
I explained.

The required changes are minor: we just need to separate the heartbeat logic from the rescheduled
timer logic, which allows the MiniYarnCluster to skip the timer.
                
> Deterministic MiniYARNCluster
> -----------------------------
>
>                 Key: YARN-1069
>                 URL: https://issues.apache.org/jira/browse/YARN-1069
>             Project: Hadoop YARN
>          Issue Type: Test
>    Affects Versions: 2.1.0-beta
>            Reporter: Maysam Yabandeh
>            Assignee: Maysam Yabandeh
>            Priority: Minor
>
> Currently the order of events in MiniYarnCluster is nondeterministic as the events are
triggered by scheduled timers. This makes it difficult to write tests since some erroneous
scenarios are triggered only after some particular sequence of events. An example is the bug
in YARN-779 where without this control over events, writing a deterministic unit test is not
possible. We should modify the source to allow both manual invocation of events as well as
via existing periodic timers. MiniYarnCluster can then expose such methods for more control
over events ordering.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message