hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2006) ability to support storing extended attributes per file
Date Thu, 24 Apr 2014 23:21:33 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13980468#comment-13980468

Andrew Purtell commented on HDFS-2006:

bq. Do you know if there is a common upper bound implemented by other file systems, like the
ext family?

FWIW, from http://en.wikipedia.org/wiki/Extended_file_attributes
For ext2/3/4 and btrfs, each extended attribute is limited to a filesystem block (e.g. 4 KiB),
and in practice in ext2/3/4 all of them must fit together on a single block (including names
and values). ReiserFS allow attributes of arbitrary size. In XFS the names can be up to 256
bytes in length, terminated by the first 0 byte, and the values can be up to 64KB of arbitrary
binary data.
In EXT4, all xattrs for a dentry must fit in the inode plus one additional block. In XFS small
attr lists are packed into the inode, large attr lists are structured externally as a btree,
with IIRC no upper limit on number. Its a large difference in design philosophy. 

> ability to support storing extended attributes per file
> -------------------------------------------------------
>                 Key: HDFS-2006
>                 URL: https://issues.apache.org/jira/browse/HDFS-2006
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: HDFS XAttrs (HDFS-2006)
>            Reporter: dhruba borthakur
>            Assignee: Yi Liu
>         Attachments: HDFS-XAttrs-Design-1.pdf, HDFS-XAttrs-Design-2.pdf, xattrs.1.patch,
> It would be nice if HDFS provides a feature to store extended attributes for files, similar
to the one described here: http://en.wikipedia.org/wiki/Extended_file_attributes. 
> The challenge is that it has to be done in such a way that a site not using this feature
does not waste precious memory resources in the namenode.

This message was sent by Atlassian JIRA

View raw message