hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1473) Refactor storage management into separate classes than fsimage file reading/writing
Date Fri, 03 Dec 2010 18:28:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966614#action_12966614
] 

Todd Lipcon commented on HDFS-1473:
-----------------------------------

I'll fix the save() -> write() momentarily.

As for the test case, there are some tests that save in parallel, but I think the race is
really narrow. U_STR is only used very briefly before it's written out to disk, so the likelihood
of threads contending is probably pretty low. The images we write in the test cases also aren't
huge - I'm more surprised that FB hasn't seen this yet on their large image tests. Hairong/Dmytro?

> Refactor storage management into separate classes than fsimage file reading/writing
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-1473
>                 URL: https://issues.apache.org/jira/browse/HDFS-1473
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>             Fix For: 0.22.0, 0.23.0
>
>         Attachments: hdfs-1473-followup.txt, hdfs-1473-prelim.txt, hdfs-1473.txt, hdfs-1473.txt,
hdfs-1473.txt
>
>
> Currently the FSImage class is responsible both for storage management (eg moving around
files, tracking file names, the VERSION file, etc) as well as for the actual serialization
and deserialization of the "fsimage" file within the storage directory.
> I'd like to refactor the loading and saving code into new classes. This will make testing
easier and also make the major changes in HDFS-1073 easier to understand.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message