hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4489) Use InodeID as as an identifier of a file in HDFS protocols and APIs
Date Sat, 27 Apr 2013 23:54:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13643839#comment-13643839
] 

Suresh Srinivas commented on HDFS-4489:
---------------------------------------

I made changes to the code to reuse the byte[][] pathComponents for file creation (made some
optimizations in that method. There are other optimizations available in terms of permission
checks that I did not venture to do). The throughput with those partial optimizations is:
||Opertaions||Elapsed||OpsPerSec||AvgTime||
|100000| 19591| 5104.384666428462| 19|
|100000| 18969| 5271.759186040382| 18|
|100000| 19206| 5206.706237634073| 19|
|100000| 18652| 5361.35535063264| 18|
|100000| 19218| 5203.455094182537| 19|
|100000| 19179| 5214.036185411127| 19|
|100000| 19302| 5180.810278727593| 19|
|100000| 19388| 5157.829585310501| 19|
|100000| 19099| 5235.876223886067| 19|
|100000| 19591| 5104.384666428462| 19|
|Average|19219.5|5204.059747|18.8|

                
> Use InodeID as as an identifier of a file in HDFS protocols and APIs
> --------------------------------------------------------------------
>
>                 Key: HDFS-4489
>                 URL: https://issues.apache.org/jira/browse/HDFS-4489
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>            Reporter: Brandon Li
>            Assignee: Brandon Li
>             Fix For: 2.0.5-beta
>
>
> The benefit of using InodeID to uniquely identify a file can be multiple folds. Here
are a few of them:
> 1. uniquely identify a file cross rename, related JIRAs include HDFS-4258, HDFS-4437.
> 2. modification checks in tools like distcp. Since a file could have been replaced or
renamed to, the file name and size combination is no t reliable, but the combination of file
id and size is unique.
> 3. id based protocol support (e.g., NFS)
> 4. to make the pluggable block placement policy use fileid instead of filename (HDFS-385).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message