hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-2718) Optimize OP_ADD in edits loading
Date Mon, 23 Jan 2012 22:53:40 GMT

     [ https://issues.apache.org/jira/browse/HDFS-2718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Konstantin Shvachko updated HDFS-2718:
--------------------------------------

    Attachment: editsLoader-trunk.patch

Here are the patches for trunk and 0.22 branch. The patch for trunk is applicable to 0.23
branch as well.
I tried to make the two patches as close to one another as possible.
No new tests included, but I modified existing tests so that they tested the new functionality.
I ran all tests for 0.22, Everything passed. For trunk I ran some key tests and will let Jenkins
validate everything else.
Merging with HA branch shouldn't be hard. In 0.22 I tested this patch with block synchronization
turned on - also works fine.
Can somebody please review this.
                
> Optimize OP_ADD in edits loading
> --------------------------------
>
>                 Key: HDFS-2718
>                 URL: https://issues.apache.org/jira/browse/HDFS-2718
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.22.0, 0.24.0, 1.0.0
>            Reporter: Konstantin Shvachko
>         Attachments: editsLoader-0.22.patch, editsLoader-trunk.patch
>
>
> During loading the edits journal FSEditLog.loadEditRecords() processes OP_ADD inefficiently.
It first removes the existing INodeFile from the directory tree, then adds it back as a regular
INodeFile, and then replaces it with INodeFileUnderConstruction if files is not closed. This
slows down edits loading. OP_ADD should be done in one shot and retain previously existing
data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message