hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zheng Shao (JIRA)" <j...@apache.org>
Subject [jira] Created: (HDFS-1127) Improve the speed of FileSystem object creation
Date Tue, 04 May 2010 04:08:55 GMT
Improve the speed of FileSystem object creation

                 Key: HDFS-1127
                 URL: https://issues.apache.org/jira/browse/HDFS-1127
             Project: Hadoop HDFS
          Issue Type: Improvement
    Affects Versions: 0.20.2, 0.20.1
            Reporter: Zheng Shao
            Assignee: Zheng Shao

It takes a long time to create a lot of {{DistributedFileSystem}} object via {{FileSystem.create}}
or {{FileSystem.createNewInstance}} function.
In our experience with scribe-hdfs, it can take up to 2 minutes for 100 such calls from each
node of a cluster of 40 nodes.

The reason for the delay is mainly because of the lock on the CACHE object. We should release
the lock while creating new FileSystem objects.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message