hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6346) Optimize OP_SET_XATTRS by persisting single Xattr entry per setXattr/removeXattr api call
Date Sat, 10 May 2014 22:12:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13992960#comment-13992960
] 

Uma Maheswara Rao G commented on HDFS-6346:
-------------------------------------------

Patch looks great!

I have few initial comments below:

{code}
 if (removedXAttr != null) {
        fsImage.getEditLog().logRemoveXAttr(src, removedXAttr);
      }
{code}
Do you think, we need warn/log to the user that xattr attempting to remove does not exist?

{code}
 XAttr unprotectedRemoveXAttr(String src,
      XAttr xAttr) throws IOException {
{code}
Need format


{code}
 XAttrStorage.updateINodeXAttrs(inode, newXAttrs, snapshotId);
    
    return existingXAttrs.size() == newXAttrs.size() ? null : xAttr;
{code}
Pls remove empty line between


When existing and updated xattrs equal then we need not even call updateINodeXAttr

so code can be changed from
{code}

{code}

to

{code}
 if (existingXAttrs.size() != newXAttrs.size()) {
      XAttrStorage.updateINodeXAttrs(inode, newXAttrs, snapshotId);
      return xAttr;
    }
    return null; {code}
?

I will continue my review tomorrow on this and post if there are any comments.

> Optimize OP_SET_XATTRS by persisting single Xattr entry per setXattr/removeXattr api
call
> -----------------------------------------------------------------------------------------
>
>                 Key: HDFS-6346
>                 URL: https://issues.apache.org/jira/browse/HDFS-6346
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Uma Maheswara Rao G
>            Assignee: Yi Liu
>         Attachments: HDFS-6346.patch, editsStored
>
>
> When a new xattrs set on an Inode, it may add this with OP_SET_XATTRS and let's say [user.name1:value1]
> On a next call if we set another xattrs, then it may store along with older existing
xattrs again. It may be like [user.name1:value1, user.name2:value2]
> So, on adding more xattrs on same Inode, that list may grow and we keep store the entries
of already existing name, value fairs.
> Right now we defaulted the max Xattrs on an Inode to 32 and configured. If user modified
it to much larger value and start setting xattrs, then edits loading may create issue like
my above example.
> But I didn't refer any usecase of having large number of xattrs, this is just the assumption
to consider a case. My biggest doubt is whether we will have such real usecases to have huge
xattrs on a single INode.
> So, here is a thought on having OP_SET_XATTR for each setXAttr operation to be logged,
When we replay them we need to consolidate. This is some initial thought we can think more
if others also feel we need to consider this case to handle.
> Otherwise we endup storing Xattrs entries in editlog file as n(n+1)/2 where n is number
xattrs for a file/dir. This may be issue only when we have large number configured max xattrs
for inode.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message