hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1127) Improve the speed of FileSystem object creation
Date Wed, 05 May 2010 06:00:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864177#action_12864177

dhruba borthakur commented on HDFS-1127:

I think Hairong mentions that HADOOP-6640 already has a fix for this one.

> Improve the speed of FileSystem object creation
> -----------------------------------------------
>                 Key: HDFS-1127
>                 URL: https://issues.apache.org/jira/browse/HDFS-1127
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 0.20.1, 0.20.2
>            Reporter: Zheng Shao
>            Assignee: Zheng Shao
>         Attachments: HDFS-1127.1.patch, HDFS-1127.branch-20.2.patch, HDFS-1127.branch-20.3.patch
> It takes a long time to create a lot of {{DistributedFileSystem}} object via {{FileSystem.create}}
or {{FileSystem.createNewInstance}} function.
> In our experience with scribe-hdfs, it can take up to 2 minutes for 100 such calls from
each node of a cluster of 40 nodes.
> The reason for the delay is mainly because of the lock on the CACHE object. We should
release the lock while creating new FileSystem objects.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message