hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4202) TaskTracker never stops cleanup threads, MiniMRCluster becomes unstable
Date Sat, 08 Nov 2008 00:43:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12645925#action_12645925

Chris Douglas commented on HADOOP-4202:

Oh, I see. Would the following idiom work for your tests? It's used in a few TestCase implementations
(TestReduceFetch, TestDatamerge, etc.) that require a cluster and run several variations:

  private static MiniMRCluster mrCluster = null;
  private static MiniDFSCluster dfsCluster = null;
  public static Test suite() {
    TestSetup setup = new TestSetup(new TestSuite(TestReduceFetch.class)) {
      protected void setUp() throws Exception {
        Configuration conf = new Configuration();
        dfsCluster = new MiniDFSCluster(conf, 2, true, null);
        mrCluster = new MiniMRCluster(2,
            dfsCluster.getFileSystem().getUri().toString(), 1);
      protected void tearDown() throws Exception {
        if (dfsCluster != null) { dfsCluster.shutdown(); }
        if (mrCluster != null) { mrCluster.shutdown(); }
    return setup;

The {{test\*}} methods all use the same, static cluster. Unless you're testing sets of parameters
required during the TT init, it should be sufficient.

> TaskTracker never stops cleanup threads, MiniMRCluster becomes unstable
> -----------------------------------------------------------------------
>                 Key: HADOOP-4202
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4202
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.18.1
>            Reporter: Chris K Wensel
>            Assignee: Chris K Wensel
>            Priority: Minor
>         Attachments: hadoop-4202.patch
> If many unit tests start/stop unique MiniMRCluster instances, over time the number of
threads in the test vm grow to large causing tests to hang and/or slow down.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message