hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7784) load fsimage in parallel
Date Fri, 13 Feb 2015 14:53:12 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Walter Su updated HDFS-7784:
----------------------------
    Priority: Minor  (was: Major)

> load fsimage in parallel
> ------------------------
>
>                 Key: HDFS-7784
>                 URL: https://issues.apache.org/jira/browse/HDFS-7784
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: Walter Su
>            Assignee: Walter Su
>            Priority: Minor
>         Attachments: HDFS-7784.001.patch, test-20150213.pdf
>
>
> When single Namenode has huge amount of files, without using federation, the startup/restart
speed is slow. The fsimage loading step takes the most of the time. fsimage loading can seperate
to two parts, deserialization and object construction(mostly map insertion). Deserialization
takes the most of CPU time. So we can do deserialization in parallel, and add to hashmap in
serial.  It will significantly reduce the NN start time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message