Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Wed, 1 Feb 2012 19:14:59 +0000 (UTC)
From: "Aaron T. Myers (Commented) (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <771015360.3824.1328123699202.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1144395578.40169.1324582651255.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-2718) Optimize OP_ADD in edits loading
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13198065#comment-13198065 ] 

Aaron T. Myers commented on HDFS-2718:
--------------------------------------

Hey Konst, the patch largely looks good. A few comments:

# Given that there's no "protected" analog to unprotectedUpdateFile, I think it should be renamed updateFile. The "unprotected" term is usually used when there's another method which calls the unprotected method, and also gets the write lock and logs to the edit log.
# Given that FSDirectory#unprotectedUpdateFile is only called from FSEditLogLoader, let's move this code to FSEditLogLoader. Then, we can also change it to take just the AddCloseOp as a parameter, instead of all the members of AddCloseOp as individual parameters.
# Nit: there's a few spots in the new patch where you have "if(". Please put a space between "if" and "(" per the style guidelines.
                
> Optimize OP_ADD in edits loading
> --------------------------------
>
>                 Key: HDFS-2718
>                 URL: https://issues.apache.org/jira/browse/HDFS-2718
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.22.0, 0.24.0, 1.0.0
>            Reporter: Konstantin Shvachko
>            Assignee: Konstantin Shvachko
>         Attachments: editsLoader-0.22.patch, editsLoader-0.22.patch, editsLoader-trunk.patch, editsLoader-trunk.patch, editsLoader-trunk.patch
>
>
> During loading the edits journal FSEditLog.loadEditRecords() processes OP_ADD inefficiently. It first removes the existing INodeFile from the directory tree, then adds it back as a regular INodeFile, and then replaces it with INodeFileUnderConstruction if files is not closed. This slows down edits loading. OP_ADD should be done in one shot and retain previously existing data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira