hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From N Keywal <nkey...@gmail.com>
Subject hdfs & flaky testFSUtils
Date Fri, 25 May 2012 23:30:05 GMT
Hello all,

Does anyone have a idea on why the code below fails? It's a
simplification of a currently flaky test in TestFSUtils.
Fails 100% of the time without the sleep, never when the sleep is set.
On paper, when the close of a hdfs file finishes we are guaranteed
that the blocks are written...


  @Test
  public void testFSUTils() throws Exception {
    final String hosts[] = {"host1", "host2", "host3", "host4"};
    Path testFile = new Path("/test1.txt");

    HBaseTestingUtility htu = new HBaseTestingUtility();

    try {
      htu.startMiniDFSCluster(hosts).waitActive();
      FileSystem fs = htu.getDFSCluster().getFileSystem();

      for (int i = 0; i < 100; ++i) {
        FSDataOutputStream out = fs.create(testFile);
        byte[] data = new byte[1];
        out.write(data, 0, 1);
        out.close();

        // Put a sleep here to make me work
        //Thread.sleep(2000);

        FileStatus status = fs.getFileStatus(testFile);
        HDFSBlocksDistribution blocksDistribution =
          FSUtils.computeHDFSBlocksDistribution(fs, status, 0, status.getLen());
        assertEquals("Wrong number of hosts distributing blocks. at
iteration "+i, 3,
          blocksDistribution.getTopHosts().size());

        fs.delete(testFile, true);
      }
    } finally {
      htu.shutdownMiniDFSCluster();
    }
  }


Thank you in advance for your help,

N.

Mime
View raw message