hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Created: (HDFS-946) NameNode should not return full path name when lisitng a diretory or getting the status of a file
Date Wed, 03 Feb 2010 20:41:27 GMT
NameNode should not return full path name when lisitng a diretory or getting the status of
a file
-------------------------------------------------------------------------------------------------

                 Key: HDFS-946
                 URL: https://issues.apache.org/jira/browse/HDFS-946
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Hairong Kuang
             Fix For: 0.22.0


FSDirectory#getListring(String src) has the following code:
      int i = 0;
      for (INode cur : contents) {
        listing[i] = createFileStatus(srcs+cur.getLocalName(), cur);
        i++;
      }
So listing a directory will return an array of FileStatus. Each FileStatus element has the
full path name. This increases the return message size and adds non-negligible CPU time to
the operation.

FSDirectory#getFileInfo(String) does not need to return the file name either.

Another optimization is that in the version of FileStatus that's used in the wire protocol,
the field path does not need to be Path; It could be a String or a byte array ideally. This
could avoid unnecessary creation of the Path objects at NameNode, thus help reduce the GC
problem observed when a large number of getFileInfo or getListing operations hit NameNode.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message