hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5568) Support inclusion of snapshot paths in Namenode fsck
Date Tue, 26 Nov 2013 22:50:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13833179#comment-13833179
] 

Jing Zhao commented on HDFS-5568:
---------------------------------

The patch looks good overall. One comment is the following code cannot cover the case that
the initial given path for fsck is under a snapshottable directory.
{code}
      if (snapshottableDirs != null && snapshottableDirs.contains(path)) {
        String snapshotPath = (path.endsWith(Path.SEPARATOR) ? path : path
            + Path.SEPARATOR)
{code}

A testcase can be:
{code}
  @Test
  public void testFsckForSnapshotFiles2() throws Exception {
    final Configuration conf = new HdfsConfiguration();
    MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).numDataNodes(1)
        .build();
    try {
      String runFsck = runFsck(conf, 0, true, "/", "-includeSnapshots",
          "-files");
      assertTrue(runFsck.contains("HEALTHY"));
      final String fileName = "/dir1/srcdat";
      DistributedFileSystem hdfs = cluster.getFileSystem();
      Path file1 = new Path(fileName);
      DFSTestUtil.createFile(hdfs, file1, 1024, (short) 1, 1000L);
      hdfs.allowSnapshot(new Path("/"));
      hdfs.createSnapshot(new Path("/"), "mySnapShot");

      runFsck = runFsck(conf, 0, true, "/", "-includeSnapshots", "-files");
      assertTrue(runFsck.contains("/.snapshot/mySnapShot/dir1/srcdat"));

      runFsck = runFsck(conf, 0, true, "/dir1", "-includeSnapshots", "-files");
      assertTrue(runFsck.contains("/.snapshot/mySnapShot/dir1/srcdat"));
    } finally {
      cluster.shutdown();
    }
  }
{code}
Thus here we may need to check if the path is under a snapshottable directory.

> Support inclusion of snapshot paths in Namenode fsck
> ----------------------------------------------------
>
>                 Key: HDFS-5568
>                 URL: https://issues.apache.org/jira/browse/HDFS-5568
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: snapshots
>            Reporter: Vinay
>            Assignee: Vinay
>         Attachments: HDFS-5568-1.patch, HDFS-5568.patch, HDFS-5568.patch
>
>
> Support Fsck to check the snapshot paths also for inconsistency.
> Currently Fsck supports snapshot paths if path given explicitly refers to a snapshot
path.
> We have seen safemode problems in our clusters which were due to blocks missing which
were only present inside snapshots. But "hdfs fsck /" shows HEALTHY. 
> So supporting snapshot paths also during fsck (may be by default or on demand) would
be helpful in these cases instead of specifying each and every snapshottable directory.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message