hadoop-hdfs-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From w...@apache.org
Subject svn commit: r1594443 - in /hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs: CHANGES-HDFS-2006.txt src/main/resources/hdfs-default.xml src/site/apt/ExtendedAttributes.apt.vm
Date Wed, 14 May 2014 01:29:37 GMT
Author: wang
Date: Wed May 14 01:29:36 2014
New Revision: 1594443

URL: http://svn.apache.org/r1594443
Log:
HDFS-6283. Write end user documentation for xattrs. (wang)

Added:
    hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
  (with props)
Modified:
    hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-2006.txt
    hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

Modified: hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-2006.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-2006.txt?rev=1594443&r1=1594442&r2=1594443&view=diff
==============================================================================
--- hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-2006.txt
(original)
+++ hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/CHANGES-HDFS-2006.txt
Wed May 14 01:29:36 2014
@@ -31,7 +31,9 @@ HDFS-2006 (Unreleased)
 
     HDFS-6377. Unify xattr name and value limits into a single limit. (wang)
 
-    HDFS-6373. Remove support for extended attributes on symlinks (Charles Lamb via wang)
+    HDFS-6373. Remove support for extended attributes on symlinks. (Charles Lamb via wang)
+
+    HDFS-6283. Write end user documentation for xattrs. (wang)
 
   OPTIMIZATIONS
 

Modified: hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
URL: http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml?rev=1594443&r1=1594442&r2=1594443&view=diff
==============================================================================
--- hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
(original)
+++ hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
Wed May 14 01:29:36 2014
@@ -1895,4 +1895,28 @@
   </description>
 </property>
 
-</configuration>
\ No newline at end of file
+<property>
+  <name>dfs.namenode.xattrs.enabled</name>
+  <value>true</value>
+  <description>
+    Whether support for extended attributes is enabled on the NameNode.
+  </description>
+</property>
+
+<property>
+  <name>dfs.namenode.fs-limits.max-xattrs-per-inode</name>
+  <value>32</value>
+  <description>
+    Maximum number of extended attributes per inode.
+  </description>
+</property>
+
+<property>
+  <name>dfs.namenode.fs-limits.max-xattr-size</name>
+  <value>16384</value>
+  <description>
+    The maximum combined size of the name and value of an extended attribute in bytes.
+  </description>
+</property>
+
+</configuration>

Added: hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
URL: http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm?rev=1594443&view=auto
==============================================================================
--- hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
(added)
+++ hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
Wed May 14 01:29:36 2014
@@ -0,0 +1,98 @@
+~~ Licensed under the Apache License, Version 2.0 (the "License");
+~~ you may not use this file except in compliance with the License.
+~~ You may obtain a copy of the License at
+~~
+~~   http://www.apache.org/licenses/LICENSE-2.0
+~~
+~~ Unless required by applicable law or agreed to in writing, software
+~~ distributed under the License is distributed on an "AS IS" BASIS,
+~~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+~~ See the License for the specific language governing permissions and
+~~ limitations under the License. See accompanying LICENSE file.
+
+  ---
+  Hadoop Distributed File System-${project.version} - Extended Attributes
+  ---
+  ---
+  ${maven.build.timestamp}
+
+Extended Attributes in HDFS
+
+  \[ {{{../../index.html}Go Back}} \]
+
+%{toc|section=1|fromDepth=2|toDepth=4}
+
+* {Overview}
+
+  <Extended attributes> (abbreviated as <xattrs>) are a filesystem feature that
allow user applications to associate additional metadata with a file or directory. Unlike
system-level inode metadata such as file permissions or modification time, extended attributes
are not interpreted by the system and are instead used by applications to store additional
information about an inode. Extended attributes could be used, for instance, to specify the
character encoding of a plain-text document.
+
+** {HDFS extended attributes}
+
+  Extended attributes in HDFS are modeled after extended attributes in Linux (see the Linux
manpage for {{{http://www.bestbits.at/acl/man/man5/attr.txt}attr(5)}} and {{{http://www.bestbits.at/acl/}related
documentation}}). An extended attribute is a <name-value pair>, with a string name and
binary value. Xattrs names must also be prefixed with a <namespace>. For example, an
xattr named <myXattr> in the <user> namespace would be specified as <<user.myXattr>>.
Multiple xattrs can be associated with a single inode.
+
+** {Namespaces and Permissions}
+
+  In HDFS, as in Linux, there are four valid namespaces: <<<user>>>, <<<trusted>>>,
<<<system>>>, and <<<security>>>. Each of these namespaces
have different access restrictions.
+
+  The <<<user>>> namespace is the namespace that will commonly be used
by client applications. Access to extended attributes in the user namespace is controlled
by the corresponding file permissions.
+
+  The <<<trusted>>> namespace is available only to HDFS superusers.
+
+  The <<<system>>> namespace is reserved for internal HDFS use. This namespace
is not accessible through userspace methods, and is reserved for implementing internal HDFS
features.
+
+  The <<<security>>> namespace is reserved for internal HDFS use. This
namespace is not accessible through userspace methods. It is currently unused.
+
+* {Interacting with extended attributes}
+
+  The Hadoop shell has support for interacting with extended attributes via <<<hadoop
fs -getfattr>>> and <<<hadoop fs -setfattr>>>. These commands are
styled after the Linux {{{http://www.bestbits.at/acl/man/man1/getfattr.txt}getfattr(1)}} and
{{{http://www.bestbits.at/acl/man/man1/setfattr.txt}setfattr(1)}} commands.
+
+** {getfattr}
+
+  <<<hadoop fs -getfattr [-R] {-n name | -d} [-e en] <path>>>>
+
+  Displays the extended attribute names and values (if any) for a file or directory.
+
+*--+--+
+-R | Recursively list the attributes for all files and directories.
+*--+--+
+-n name | Dump the named extended attribute value.
+*--+--+
+-d | Dump all extended attribute values associated with pathname.
+*--+--+
+-e \<encoding\> | Encode values after retrieving them. Valid encodings are "text",
"hex", and "base64". Values encoded as text strings are enclosed in double quotes ("), and
values encoded as hexadecimal and base64 are prefixed with 0x and 0s, respectively.
+*--+--+
+\<path\> | The file or directory.
+*--+--+
+
+** {setfattr}
+
+  <<<hadoop fs -setfattr {-n name [-v value] | -x name} <path>>>>
+
+  Sets an extended attribute name and value for a file or directory.
+
+*--+--+
+-n name | The extended attribute name.
+*--+--+
+-v value | The extended attribute value. There are three different encoding methods for the
value. If the argument is enclosed in double quotes, then the value is the string inside the
quotes. If the argument is prefixed with 0x or 0X, then it is taken as a hexadecimal number.
If the argument begins with 0s or 0S, then it is taken as a base64 encoding.
+*--+--+
+-x name | Remove the extended attribute.
+*--+--+
+\<path\> | The file or directory.
+*--+--+
+
+* {Configuration options}
+
+
+  HDFS supports extended attributes out of the box, without additional configuration. Administrators
could potentially be interested in the options limiting the number of xattrs per inode and
the size of xattrs, since xattrs increase the on-disk and in-memory space consumption of an
inode.
+
+  * <<<dfs.namenode.xattrs.enabled>>>
+
+  Whether support for extended attributes is enabled on the NameNode. By default, extended
attributes are enabled.
+
+  * <<<dfs.namenode.fs-limits.max-xattrs-per-inode>>>
+
+  The maximum number of extended attributes per inode. By default, this limit is 32.
+
+  * <<<dfs.namenode.fs-limits.max-xattr-size>>>
+
+  The maximum combined size of the name and value of an extended attribute in bytes. By default,
this limit is 16384 bytes.

Propchange: hadoop/common/branches/HDFS-2006/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
------------------------------------------------------------------------------
    svn:eol-style = native



Mime
View raw message