hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Created: (HDFS-1070) Speedup NameNode image loading and saving
Date Mon, 29 Mar 2010 18:13:27 GMT
Speedup NameNode image loading and saving

                 Key: HDFS-1070
                 URL: https://issues.apache.org/jira/browse/HDFS-1070
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: name-node
            Reporter: Hairong Kuang
            Assignee: Hairong Kuang
             Fix For: 0.22.0

Currently each inode stores its full path in the fsimage. I'd propose to store the local name
instead. In order for each inode to identify its parent, all inodes in a directory tree are
stored in the image in in-order. This proposal also requires each directory stores the number
of its children in image.

This proposal would bring a few benefits as pointed below and therefore speedup the image
loading and saving.
1. Remove the overhead of converting java-UTF8 encoded local name to string-represented full
path then to UTF8 encoded full path when saving to image and vice versa when loading the image.
2. Remove the overhead of traversing the full path when inserting the inode to its parent
2. Reduce the number of temporary java objects during the process of image loading or saving
and  therefore reduce the GC overhead.
3. Reduce the size of image.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message