Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 1 Aug 2012 11:43:03 +0000 (UTC)
From: "Vinay (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <1594578486.303.1343821383526.JavaMail.jiratomcat@issues-vm>
In-Reply-To: <1991555652.115623.1343639074826.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Commented] (HDFS-3734)
 TestFSEditLogLoader.testReplicationAdjusted() will hang if number of blocks
 are more than one
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426537#comment-13426537 ] 

Vinay commented on HDFS-3734:
-----------------------------

Here is the modified test case
{code}@Test(timeout = 60000)
  public void testReplicationAdjusted() throws IOException {
    // start a cluster 
    Configuration conf = new HdfsConfiguration();
    // Replicate and heartbeat fast to shave a few seconds off test
    conf.setInt(DFSConfigKeys.DFS_NAMENODE_REPLICATION_INTERVAL_KEY, 1);
    conf.setInt(DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY, 1);

    MiniDFSCluster cluster = null;
    try {
      cluster = new MiniDFSCluster.Builder(conf).numDataNodes(2)
          .build();
      cluster.waitActive();
      FileSystem fs = cluster.getFileSystem();
  
      // Create a file with replication count 1
      Path p = new Path("/testfile");
      DFSTestUtil.createFile(fs, p, 10, /*repl*/ (short)1, 1);
      DFSTestUtil.waitReplication(fs, p, (short)1);

      Path p2 = new Path("/testfile2");
      DFSTestUtil.createFile(fs, p2, 10, /*repl*/ (short)1, 1);
      DFSTestUtil.waitReplication(fs, p2, (short)1);
  
      // Shut down and restart cluster with new minimum replication of 2
      cluster.shutdown();
      cluster = null;
      
      conf.setInt(DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_KEY, 2);
  
      cluster = new MiniDFSCluster.Builder(conf).numDataNodes(2)
        .format(false).build();
      cluster.waitActive();
      fs = cluster.getFileSystem();
      
      // The file should get adjusted to replication 2 when
      // the edit log is replayed.
      DFSTestUtil.waitReplication(fs, p, (short)2);
    } finally {
      if (cluster != null) {
        cluster.shutdown();
      }
    }
  }{code}
                
> TestFSEditLogLoader.testReplicationAdjusted() will hang if number of blocks are more than one
> ---------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3734
>                 URL: https://issues.apache.org/jira/browse/HDFS-3734
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.1.0-alpha, 3.0.0
>            Reporter: Vinay
>
> TestFSEditLogLoader.testReplicationAdjusted() which was added in HDFS-2003 will fail if number of blocks before cluster restart are more than one.
> Test Scenario:
> --------------
> 1. Write a file with min replication as 1 and replication factor as 1.
> 2. Change the min replication to 2 and restart the cluster.
> Expected: Min replication should be automatically reset on cluster restart by replicating more blocks.
> Currently, if the number of blocks before restart is only one, then on restart NN will not enter safemode, hence replication will happen and satisfies min replication factor.
> If initial blocks count is more than 1 which are having replication factor as 1, then on restart NN will enter safemode and will never come out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira