hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9621) getListing wrongly associates Erasure Coding policy to pre-existing replicated files under an EC directory
Date Thu, 07 Jan 2016 03:50:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086760#comment-15086760
] 

Kai Zheng commented on HDFS-9621:
---------------------------------

Thanks for the work Jing. The patch looks great. Some minor comments:
1. For this change:
{code}
+      ecPolicy = fileNode.isStriped() ? ecPolicy : null;
{code}
How about:
{code}
ecPolicy = null;
if (fileNode.isStriped()) {
  ecPolicy = FSDirErasureCodingOp.getErasureCodingPolicy(fsd.getFSNamesystem(), iip);
}
{code}

2. For this codes:
{code}
+    DirectoryListing listing = fs.getClient().listPaths(dir.toString(),
+        new byte[0], false);
+    HdfsFileStatus[] files = listing.getPartialListing();
+    assertNotNull(files[0].getErasureCodingPolicy()); // ecSubDir
+    assertNull(files[1].getErasureCodingPolicy()); // replicatedFile
{code}
Might be not very reliable relying on the listed entry order considering {{listPaths}} or
{{getPartialListing}} may change in implementation.

> getListing wrongly associates Erasure Coding policy to pre-existing replicated files
under an EC directory  
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-9621
>                 URL: https://issues.apache.org/jira/browse/HDFS-9621
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: erasure-coding
>    Affects Versions: 3.0.0
>            Reporter: Sushmitha Sreenivasan
>            Assignee: Jing Zhao
>            Priority: Critical
>         Attachments: HDFS-9621.000.patch
>
>
> This is reported by [~ssreenivasan]:
> If we set Erasure Coding policy to a directory which contains some files with replicated
blocks, later when listing files under the directory these files will be reported as EC files.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message