hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HDFS-16187) SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with checkpointing
Date Fri, 10 Sep 2021 03:20:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-16187?focusedWorklogId=649008&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-649008
]

ASF GitHub Bot logged work on HDFS-16187:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 10/Sep/21 03:19
            Start Date: 10/Sep/21 03:19
    Worklog Time Spent: 10m 
      Work Description: bshashikant edited a comment on pull request #3340:
URL: https://github.com/apache/hadoop/pull/3340#issuecomment-916599655


   
   Thanks @cnauroth . The problem here is very specific to snapshotRoot. 
   
   `@Test
     public void testXattrOnFileWithSnapshotAndNNRestart() throws Exception {
       // Init
       FileSystem.mkdirs(hdfs, path, FsPermission.createImmutable((short) 0700));
       Path file = new Path(path, "file1");
       FileSystem.create(hdfs, file, FsPermission.createImmutable((short) 07777));
       hdfs.setXAttr(file, name1, value1);
       hdfs.allowSnapshot(path);
       hdfs.createSnapshot(path, snapshotName);
       SnapshotDiffReport report =
           hdfs.getSnapshotDiffReport(path, snapshotName, "");
       System.out.println(report);
       Assert.assertEquals(0, report.getDiffList().size());
       report =
           hdfs.getSnapshotDiffReport(path, snapshotName, "");
       System.out.println(report);
       Assert.assertEquals(0, report.getDiffList().size());
       hdfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_ENTER);
       hdfs.saveNamespace();
       hdfs.setSafeMode(HdfsConstants.SafeModeAction.SAFEMODE_LEAVE);
       cluster.restartNameNode(true);
       report =
           hdfs.getSnapshotDiffReport(path, snapshotName, "");
       System.out.println(report);
       Assert.assertEquals(0, report.getDiffList().size());
     }`
   
   The same test with xattrs set on file inside the snapshotRoot/SubDirecory will just work
with/without the patch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 649008)
    Time Spent: 2h 10m  (was: 2h)

> SnapshotDiff behaviour with Xattrs and Acls is not consistent across NN restarts with
checkpointing
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-16187
>                 URL: https://issues.apache.org/jira/browse/HDFS-16187
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: snapshots
>            Reporter: Srinivasu Majeti
>            Assignee: Shashikant Banerjee
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> The below test shows the snapshot diff between across snapshots is not consistent with
Xattr(EZ here settinh the Xattr) across NN restarts with checkpointed FsImage.
> {code:java}
> @Test
> public void testEncryptionZonesWithSnapshots() throws Exception {
>   final Path snapshottable = new Path("/zones");
>   fsWrapper.mkdir(snapshottable, FsPermission.getDirDefault(),
>       true);
>   dfsAdmin.allowSnapshot(snapshottable);
>   dfsAdmin.createEncryptionZone(snapshottable, TEST_KEY, NO_TRASH);
>   fs.createSnapshot(snapshottable, "snap1");
>   SnapshotDiffReport report =
>       fs.getSnapshotDiffReport(snapshottable, "snap1", "");
>   Assert.assertEquals(0, report.getDiffList().size());
>   report =
>       fs.getSnapshotDiffReport(snapshottable, "snap1", "");
>   System.out.println(report);
>   Assert.assertEquals(0, report.getDiffList().size());
>   fs.setSafeMode(SafeModeAction.SAFEMODE_ENTER);
>   fs.saveNamespace();
>   fs.setSafeMode(SafeModeAction.SAFEMODE_LEAVE);
>   cluster.restartNameNode(true);
>   report =
>       fs.getSnapshotDiffReport(snapshottable, "snap1", "");
>   Assert.assertEquals(0, report.getDiffList().size());
> }{code}
> {code:java}
> Pre Restart:
> Difference between snapshot snap1 and current directory under directory /zones:
> Post Restart:
> Difference between snapshot snap1 and current directory under directory /zones:
> M .{code}
> The side effect of this behavior is : distcp with snapshot diff would fail with below
error complaining that target cluster has some data changed .
> {code:java}
> WARN tools.DistCp: The target has been modified since snapshot xxxxx
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message