hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3734) TestFSEditLogLoader.testReplicationAdjusted() will hang if number of blocks are more than one
Date Wed, 01 Aug 2012 11:43:03 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426537#comment-13426537
] 

Vinay commented on HDFS-3734:
-----------------------------

Here is the modified test case
{code}@Test(timeout = 60000)
  public void testReplicationAdjusted() throws IOException {
    // start a cluster 
    Configuration conf = new HdfsConfiguration();
    // Replicate and heartbeat fast to shave a few seconds off test
    conf.setInt(DFSConfigKeys.DFS_NAMENODE_REPLICATION_INTERVAL_KEY, 1);
    conf.setInt(DFSConfigKeys.DFS_HEARTBEAT_INTERVAL_KEY, 1);

    MiniDFSCluster cluster = null;
    try {
      cluster = new MiniDFSCluster.Builder(conf).numDataNodes(2)
          .build();
      cluster.waitActive();
      FileSystem fs = cluster.getFileSystem();
  
      // Create a file with replication count 1
      Path p = new Path("/testfile");
      DFSTestUtil.createFile(fs, p, 10, /*repl*/ (short)1, 1);
      DFSTestUtil.waitReplication(fs, p, (short)1);

      Path p2 = new Path("/testfile2");
      DFSTestUtil.createFile(fs, p2, 10, /*repl*/ (short)1, 1);
      DFSTestUtil.waitReplication(fs, p2, (short)1);
  
      // Shut down and restart cluster with new minimum replication of 2
      cluster.shutdown();
      cluster = null;
      
      conf.setInt(DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_KEY, 2);
  
      cluster = new MiniDFSCluster.Builder(conf).numDataNodes(2)
        .format(false).build();
      cluster.waitActive();
      fs = cluster.getFileSystem();
      
      // The file should get adjusted to replication 2 when
      // the edit log is replayed.
      DFSTestUtil.waitReplication(fs, p, (short)2);
    } finally {
      if (cluster != null) {
        cluster.shutdown();
      }
    }
  }{code}
                
> TestFSEditLogLoader.testReplicationAdjusted() will hang if number of blocks are more
than one
> ---------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3734
>                 URL: https://issues.apache.org/jira/browse/HDFS-3734
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.1.0-alpha, 3.0.0
>            Reporter: Vinay
>
> TestFSEditLogLoader.testReplicationAdjusted() which was added in HDFS-2003 will fail
if number of blocks before cluster restart are more than one.
> Test Scenario:
> --------------
> 1. Write a file with min replication as 1 and replication factor as 1.
> 2. Change the min replication to 2 and restart the cluster.
> Expected: Min replication should be automatically reset on cluster restart by replicating
more blocks.
> Currently, if the number of blocks before restart is only one, then on restart NN will
not enter safemode, hence replication will happen and satisfies min replication factor.
> If initial blocks count is more than 1 which are having replication factor as 1, then
on restart NN will enter safemode and will never come out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message