hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2423) The codes in FSDirectory.mkdirs(...) is inefficient.
Date Mon, 10 Mar 2008 20:08:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12577154#action_12577154

Konstantin Shvachko commented on HADOOP-2423:

This looks much better. There is room for more optimization.
# Rather than calling rootDir.addNode() you should call parentDir.addChild().
This is what essentially happens in your code.
But doing this explicitely will let you avoid creating the new addNode() with redundant parameters.
#    String[] strings = path.split(Path.SEPARATOR, -1);
why limit = -1 in the current code is replaced by 0?
#  <T extends INode> T addNode(String path, byte[][] pathComponents, INode parentNode,
T newNode, boolean inheritPermission)
parentNode should be of type INodeDirectory
String path, don't need the first parameter:
return null rather than throwing an exception; and throw FileNotFoundException in the calling
# In FSDirectory.mkdirs() the FileNotFoundException thrown by addNode() is absorbed. Should
it be?

> The codes in FSDirectory.mkdirs(...) is inefficient.
> ----------------------------------------------------
>                 Key: HADOOP-2423
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2423
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.15.1
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>         Attachments: 2423_20080130.patch, 2423_20080303.patch, 2423_20080304.patch, 2423_20080304b.patch,
2423_20080304c.patch, 2423_20080304d.patch
> FSDirectory.mkdirs(...) creates List<String> v to store all dirs.  e.g.
> {code}
> //Suppose 
> src = "/foo/bar/bas/"
> //Then,
> v = {"/", "/foo", "/foo/bar", "/foo/bar/bas"}
> {code}
> For each directory string *cur* in v, no matter *cur* already exists or not, it will
try to do a unprotectedMkdir(cur, ...).  Then, *cur* is parsed to byte[][] in INodeDirectory.addNode
> We don't need to do the parsing for each string in v.  Instead, byte[][] should be stored.
 Also, the loop should not continue once it finds an existing subdirectory.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message