hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2718) Optimize OP_ADD in edits loading
Date Wed, 01 Feb 2012 19:14:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198065#comment-13198065
] 

Aaron T. Myers commented on HDFS-2718:
--------------------------------------

Hey Konst, the patch largely looks good. A few comments:

# Given that there's no "protected" analog to unprotectedUpdateFile, I think it should be
renamed updateFile. The "unprotected" term is usually used when there's another method which
calls the unprotected method, and also gets the write lock and logs to the edit log.
# Given that FSDirectory#unprotectedUpdateFile is only called from FSEditLogLoader, let's
move this code to FSEditLogLoader. Then, we can also change it to take just the AddCloseOp
as a parameter, instead of all the members of AddCloseOp as individual parameters.
# Nit: there's a few spots in the new patch where you have "if(". Please put a space between
"if" and "(" per the style guidelines.
                
> Optimize OP_ADD in edits loading
> --------------------------------
>
>                 Key: HDFS-2718
>                 URL: https://issues.apache.org/jira/browse/HDFS-2718
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.22.0, 0.24.0, 1.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>         Attachments: editsLoader-0.22.patch, editsLoader-0.22.patch, editsLoader-trunk.patch,
editsLoader-trunk.patch, editsLoader-trunk.patch
>
>
> During loading the edits journal FSEditLog.loadEditRecords() processes OP_ADD inefficiently.
It first removes the existing INodeFile from the directory tree, then adds it back as a regular
INodeFile, and then replaces it with INodeFileUnderConstruction if files is not closed. This
slows down edits loading. OP_ADD should be done in one shot and retain previously existing
data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message