hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11084) OIV ReverseXML processor does not recognize sticky bit
Date Tue, 13 Dec 2016 18:39:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15745883#comment-15745883

Wei-Chiu Chuang commented on HDFS-11084:

For the context, OIV is a tool that dumps fsimage into a human-readable file, such as XML.
OIV ReverseXML is a tool that imports an XML file into a fsimage.

So ideally, you can do fsimage --> XML --> fsimage, and the sourcee fsimage is exactly
the same as the destination fsimage. However, in branch-2 if the fsimage contains directories
with sticky bits, it fails with the following exception:

Invalid processor specified : invalid
OfflineImageReconstructor failed: 1777
java.lang.IllegalArgumentException: 1777
	at org.apache.hadoop.fs.permission.PermissionParser.<init>(PermissionParser.java:60)
	at org.apache.hadoop.fs.permission.UmaskParser.<init>(UmaskParser.java:42)
	at org.apache.hadoop.fs.permission.FsPermission.<init>(FsPermission.java:106)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.permissionXmlToU64(OfflineImageReconstructor.java:1486)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor.java:676)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processINodeXml(OfflineImageReconstructor.java:555)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.access$1300(OfflineImageReconstructor.java:95)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeSectionProcessor.process(OfflineImageReconstructor.java:526)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processXml(OfflineImageReconstructor.java:1658)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(OfflineImageReconstructor.java:1713)
	at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewerPB.run(OfflineImageViewerPB.java:191)

Once I backport HADOOP-13508, this exception goes away. However, given that HADOOP-13508 is
an incompatible change, I doubt people want to backport it just to fix this bug. So I propose
FsPermission implements a similar helper method (which implements the semantics in trunk)
and let OfflineImageReconstructor call that helper method.

Attach a patch to reproduce the bug.

> OIV ReverseXML processor does not recognize sticky bit
> ------------------------------------------------------
>                 Key: HDFS-11084
>                 URL: https://issues.apache.org/jira/browse/HDFS-11084
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: tools
>    Affects Versions: 2.8.0
>            Reporter: Wei-Chiu Chuang
>         Attachments: HDFS-11804.branch-2.patch
> HDFS-10505 added a new feature OIV ReverseXML processor to generate a fsimage from a
xml file. However, if the files/directories in it has sticky bit, ReverseXML processor can
not recognize it due to HADOOP-13508.
> It seems HADOOP-13508 is an incompatible change in Hadoop 3. Would it be reasonable to
add an overloaded FsPermission constructor that uses RawParser so that it reads sticky bits
correctly? Or is it reasonable to backport HADOOP-13508 to branch-2?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message