hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles Lamb (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10520) Extended attributes definition and FileSystem APIs for extended attributes.
Date Mon, 28 Apr 2014 13:05:18 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13982992#comment-13982992
] 

Charles Lamb commented on HADOOP-10520:
---------------------------------------

Hi,

I read over the patch and it looks good. 

Should the indentation be 2, not 4? There are a few places where 4 is used and I thought the
coding standard was 2.

I've gone over the javadoc and "unified" it and cleaned up some typos. If it's easier for
you to incorporate the changes  below I can upload a file with these suggested changes.

Charles

Index: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
===================================================================
--- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
(revision 1
589028)
+++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
(working co
py)
@@ -1038,7 +1038,155 @@
+   * Set the xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system which 
+   * followed by ".", For example "user.attr".
+   * <p/>
+   * A regular user only can set xattr of "user" namespace.
+   * A super user can set xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
+   * For xattr of "user" namespace, its access permissions are 
+   * defined by the file or directory permission bits.
+   * XAttr will be set only when login user has correct permissions.
+   * If the xattr exists, it will be replaced.

Set an xattr of a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>
A regular user can only set an xattr for the "user" namespace.
The super user can set an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
The access permissions of an xattr in the "user" namespace are
defined by the file and directory permission bits.
An xattr can only be set when the logged user has the correct permissions.
If the xattr exists, it will be replaced.

+  public void setXAttr(Path path, String name, byte[] value) 
+      throws IOException {
+    setXAttr(path, name, value, EnumSet.of(XAttrSetFlag.CREATE, 
+        XAttrSetFlag.REPLACE));

+   * Set the xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system which 
+   * followed by ".", For example "user.attr".
+   * <p/>
+   * A regular user only can set xattr of "user" namespace.
+   * A super user can set xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
[cwl@localhost patches]$ cat HADOOP-10520-comments.txt 
Should the indentation be 2, not 4?

Index: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
===================================================================
--- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
(revision 1589028)
+++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java
(working copy)
@@ -1038,7 +1038,155 @@
+   * Set the xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system which 
+   * followed by ".", For example "user.attr".
+   * <p/>
+   * A regular user only can set xattr of "user" namespace.
+   * A super user can set xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
+   * For xattr of "user" namespace, its access permissions are 
+   * defined by the file or directory permission bits.
+   * XAttr will be set only when login user has correct permissions.
+   * If the xattr exists, it will be replaced.

Set an xattr of a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>
A regular user can only set an xattr for the "user" namespace.
The super user can set an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
The access permissions of an xattr in the "user" namespace are
defined by the file and directory permission bits.
An xattr can only be set when the logged user has the correct permissions.
If the xattr exists, it will be replaced.

+  public void setXAttr(Path path, String name, byte[] value) 
+      throws IOException {
+    setXAttr(path, name, value, EnumSet.of(XAttrSetFlag.CREATE, 
+        XAttrSetFlag.REPLACE));

+   * Set the xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system which 
+   * followed by ".", For example "user.attr".
+   * <p/>
+   * A regular user only can set xattr of "user" namespace.
+   * A super user can set xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
+   * For xattr of "user" namespace, its access permissions are 
+   * defined by the file or directory permission bits.
+   * XAttr will be set only when login user has correct permissions.

Set an xattr of a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>
A regular user can only set an xattr for the "user" namespace.
The super user can set an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
The access permissions of an xattr in the "user" namespace are
defined by the file and directory permission bits.
An xattr can only be set when the logged user has the correct permissions.
If the xattr exists, it will be replaced.

+  public void setXAttr(Path path, String name, byte[] value, 
+      EnumSet<XAttrSetFlag> flag) throws IOException {
+    throw new UnsupportedOperationException(getClass().getSimpleName()
+        + " doesn't support setXAttr");

+   * Get xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system.
+   * <p/>
+   * A regular user only can get xattr of "user" namespace.
+   * A super user can get xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
+   * XAttr will be returned only when login user has correct permissions.
+   * <p/>

Get an xattr for a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>

A regular user can only get an xattr for the "user" namespace.
The super user can get an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
An xattr will only be returned when the logged-in user has the correct permissions.
<p/>

+  public byte[] getXAttr(Path path, String name) throws IOException {
+    throw new UnsupportedOperationException(getClass().getSimpleName()
+        + " doesn't support getXAttr");

+  }
+  
+  /**
+   * Get all xattrs of a file or directory.
+   * Only xattrs which login user has correct permissions will be returned.
+   * <p/>
+   * A regular user only can get xattr of "user" namespace.
+   * A super user can get xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.

Get all of the xattrs for a file or directory.
Only those xattrs for which the logged-in user has permissions to view
are returned.
<p/>
A regular user can only get xattrs for the "user" namespace.
The super user can only get xattrs for "user" and "trusted" namespaces.
The xattr of "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.

+  public Map<String, byte[]> getXAttrs(Path path) throws IOException {
+    throw new UnsupportedOperationException(getClass().getSimpleName()
+        + " doesn't support getXAttrs");

+   * Get the xattrs of a file or directory.
+   * Name must be prefixed with user/trusted/security/system.
+   * Only xattrs which login user has correct permissions will be returned.
+   * <p/>
+   * A regular user only can get xattr of "user" namespace.
+   * A super user can get xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.

Get all of the xattrs for a file or directory.
Only those xattrs for which the logged-in user has permissions to view
are returned.
<p/>
A regular user can only get xattrs for the "user" namespace.
The super user can only get xattrs for "user" and "trusted" namespaces.
The xattr of "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.

+  public Map<String, byte[]> getXAttrs(Path path, List<String> names) 
+      throws IOException {
+    throw new UnsupportedOperationException(getClass().getSimpleName()
+        + " doesn't support getXAttrs");

+   * Remove xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system.
+   * <p/>
+   * A regular user only can remove xattr of "user" namespace.
+   * A super user can remove xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.

Remove an xattr of a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>
A regular user can only remove an xattr for the "user" namespace.
The super user can remove an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
The access permissions of an xattr in the "user" namespace are
defined by the file and directory permission bits.
An xattr can only be set when the logged user has the correct permissions.
If the xattr exists, it will be replaced.

+  public void removeXAttr(Path path, String name) throws IOException {
+    throw new UnsupportedOperationException(getClass().getSimpleName()
+        + " doesn't support removeXAttr");

Index: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
===================================================================
--- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
(revision 1589028)
+++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileContext.java
(working copy)
@@ -2294,4 +2294,184 @@

+   * Set the xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system which 
+   * followed by ".", For example "user.attr".
+   * <p/>
+   * A regular user only can set xattr of "user" namespace.
+   * A super user can set xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
+   * For xattr of "user" namespace, its access permissions are 
+   * defined by the file or directory permission bits.
+   * XAttr will be set only when login user has correct permissions.
+   * If the xattr exists, it will be replaced.

Set an xattr of a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>
A regular user can only set an xattr for the "user" namespace.
The super user can set an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
The access permissions of an xattr in the "user" namespace are
defined by the file and directory permission bits.
An xattr can only be set when the logged user has the correct permissions.
If the xattr exists, it will be replaced.

+  public void setXAttr(Path path, String name, byte[] value) 
+      throws IOException {
+    setXAttr(path, name, value, EnumSet.of(XAttrSetFlag.CREATE, 
+        XAttrSetFlag.REPLACE));
+  }

+   * Set the xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system which 
+   * followed by ".", For example "user.attr".
+   * <p/>
+   * A regular user only can set xattr of "user" namespace.
+   * A super user can set xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
+   * For xattr of "user" namespace, its access permissions are 
+   * defined by the file or directory permission bits.
+   * XAttr will be set only when login user has correct permissions.

Set an xattr of a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>
A regular user can only set an xattr for the "user" namespace.
The super user can set an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
The access permissions of an xattr in the "user" namespace are
defined by the file and directory permission bits.
An xattr can only be set when the logged user has the correct permissions.
If the xattr exists, it will be replaced.

+  public void setXAttr(Path path, final String name, final byte[] value, 
+      final EnumSet<XAttrSetFlag> flag) throws IOException {
+    Path absF = fixRelativePart(path);

Consider a "final" decl?

+    new FSLinkResolver<Void>() {
+      @Override
+      public Void next(final AbstractFileSystem fs, final Path p)
+          throws IOException {
+        fs.setXAttr(p, name, value, flag);
+        return null;
+      }
+    }.resolve(this, absF);
+  }
+  
+  /**
+   * Get xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system.
+   * <p/>
+   * A regular user only can get xattr of "user" namespace.
+   * A super user can get xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.
+   * <p/>
+   * XAttr will be returned only when login user has correct permissions.

Get an xattr for a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>

A regular user can only get an xattr for the "user" namespace.
The super user can get an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
An xattr will only be returned when the logged-in user has the correct permissions.
<p/>

+   */
+  public byte[] getXAttr(Path path, final String name) throws IOException {
+    Path absF = fixRelativePart(path);

Consider a "final" decl?

+    return new FSLinkResolver<byte[]>() {
+      @Override
+      public byte[] next(final AbstractFileSystem fs, final Path p)
+          throws IOException {
+        return fs.getXAttr(p, name);
+      }
+    }.resolve(this, absF);
+  }
+  
+  /**
+   * Get all xattrs of a file or directory.
+   * Only xattrs which login user has correct permissions will be returned. 
+   * <p/>
+   * A regular user only can get xattr of "user" namespace.
+   * A super user can get xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.

Get all of the xattrs for a file or directory.
Only those xattrs for which the logged-in user has permissions to view
are returned.
<p/>
A regular user can only get xattrs for the "user" namespace.
The super user can only get xattrs for "user" and "trusted" namespaces.
The xattr of "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.

+  public Map<String, byte[]> getXAttrs(Path path) throws IOException {
+    Path absF = fixRelativePart(path);

Consider a "final" decl?

+    return new FSLinkResolver<Map<String, byte[]>>() {
+      @Override
+      public Map<String, byte[]> next(final AbstractFileSystem fs, final Path p)
+          throws IOException {
+        return fs.getXAttrs(p);
+      }
+    }.resolve(this, absF);
+  }
+  
+  /**
+   * Get the xattrs of a file or directory.
+   * Name must be prefixed with user/trusted/security/system.
+   * Only xattrs which login user has correct permissions will be returned.
+   * <p/>
+   * A regular user only can get xattr of "user" namespace.
+   * A super user can get xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.

Get all of the xattrs for a file or directory.
Only those xattrs for which the logged-in user has permissions to view
are returned.
<p/>
A regular user can only get xattrs for the "user" namespace.
The super user can only get xattrs for "user" and "trusted" namespaces.
The xattr of "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.

+  public Map<String, byte[]> getXAttrs(Path path, final List<String> names) 
+      throws IOException {
+    Path absF = fixRelativePart(path);

Consider a "final" decl?

+    return new FSLinkResolver<Map<String, byte[]>>() {
+      @Override
+      public Map<String, byte[]> next(final AbstractFileSystem fs, final Path p)
+          throws IOException {
+        return fs.getXAttrs(p, names);
+      }
+    }.resolve(this, absF);
+  }
+  
+  /**
+   * Remove xattr of a file or directory.
+   * Name must be prefixed with user/trusted/security/system.
+   * <p/>
+   * A regular user only can remove xattr of "user" namespace.
+   * A super user can remove xattr of "user" and "trusted" namespace.
+   * XAttr of "security" and "system" namespace is only used/exposed 
+   * internally to the FS impl.

Remove an xattr of a file or directory.
The name must be prefixed with user/trusted/security/system and
followed by ".". For example, "user.attr".
<p/>
A regular user can only remove an xattr for the "user" namespace.
The super user can remove an xattr of either the "user" or "trusted" namespaces.
The xattrs of the "security" and "system" namespaces are only used/exposed 
internally by/to the FS impl.
<p/>
The access permissions of an xattr in the "user" namespace are
defined by the file and directory permission bits.
An xattr can only be set when the logged user has the correct permissions.
If the xattr exists, it will be replaced.

+  public void removeXAttr(Path path, final String name) throws IOException {
+    Path absF = fixRelativePart(path);

Consider a "final" decl?

+    new FSLinkResolver<Void>() {
+      @Override
+      public Void next(final AbstractFileSystem fs, final Path p)
+          throws IOException {
+        fs.removeXAttr(p, name);
+        return null;
+      }
+    }.resolve(this, absF);
+  }
 }
Index: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
===================================================================
--- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
(revision 1589028)
+++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
(working copy)
@@ -2349,6 +2349,154 @@

Same javadoc comments as FileContext.java above.

Index: hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/XAttrSetFlag.java
===================================================================
--- hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/XAttrSetFlag.java
(revision 0)
+++ hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/XAttrSetFlag.java
(working copy)
@@ -0,0 +1,71 @@

+  public static void validate(String xAttrName, boolean xAttrExists, 
+      EnumSet<XAttrSetFlag> flag) throws IOException {
+    if (flag == null || flag.isEmpty()) {
+      throw new HadoopIllegalArgumentException("Should specify set flag");

s/"Should specify set flag"/"A flag must be specified."

+    }
+
+    if (xAttrExists) {
+      if (!flag.contains(REPLACE)) {
+        throw new IOException("XAttr: " + xAttrName + 
+            " already exists. REPLACE flag must be specified.");

s/REPLACE/The REPLACE/

+      }
+    } else {
+      if (!flag.contains(CREATE)) {
+        throw new IOException("XAttr: " + xAttrName + 
+            " does not exist. CREATE flag must be specified.");

s/CREATE/The CREATE/

+      }
+    }
+  }
+}


> Extended attributes definition and FileSystem APIs for extended attributes.
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-10520
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10520
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs
>    Affects Versions: HDFS XAttrs (HDFS-2006)
>            Reporter: Yi Liu
>            Assignee: Yi Liu
>             Fix For: 3.0.0
>
>         Attachments: HADOOP-10520.1.patch, HADOOP-10520.2.patch, HADOOP-10520.3.patch,
HADOOP-10520.4.patch, HADOOP-10520.5.patch, HADOOP-10520.6.patch, HADOOP-10520.patch
>
>
> This JIRA defines XAttr (Extended Attribute), it consists of a name and associated data,
and 4 namespaces are defined: user, trusted, security and system. FileSystem APIs for XAttr
include setXAttrs, getXAttrs, removeXAttrs and so on. For more information, please refer to
HDFS-2006.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message