Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 27 May 2011 07:00:47 +0000 (UTC)
From: "Aaron T. Myers (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <727110783.47979.1306479647830.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <494112253.47725.1306470407502.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-2006) ability to support storing extended
 attributes per file
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-2006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040092#comment-13040092 ] 

Aaron T. Myers commented on HDFS-2006:
--------------------------------------

One option to not waste memory resources for files/directories which do not have extended attributes set on them would be to go the same route as the {{INodeDirectory}} vs. {{INodeDirectoryWithQuota}} classes, i.e. have {{INodeFile}} and {{InodeFileWithXAttrs}}. Of course, this strategy requires that we implement classes for the matrix of all the different possible combinations of extra metadata types.

A reasonable question to ask is whether we should concern ourselves all that much with the memory consumption of this, since the extended attributes will presumably be per-file. Since the NN's memory is dedicated to relatively few files and relatively many blocks, it may not matter much if we go with a fairly naive approach.

Dhruba, do you have any data as to what percentage of the NN heap in a typical cluster goes to per-file metadata versus per-block metadata?

> ability to support storing extended attributes per file
> -------------------------------------------------------
>
>                 Key: HDFS-2006
>                 URL: https://issues.apache.org/jira/browse/HDFS-2006
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> It would be nice if HDFS provides a feature to store extended attributes for files, similar to the one described here: http://en.wikipedia.org/wiki/Extended_file_attributes. 
> The challenge is that it has to be done in such a way that a site not using this feature does not waste precious memory resources in the namenode.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira