hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4404) saveFSImage() should remove files from a storage directory that do not correspond to its type.
Date Mon, 13 Oct 2008 22:08:44 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Konstantin Shvachko updated HADOOP-4404:

    Issue Type: Bug  (was: Improvement)

After successfully splitting directory {{name}}, which contained both {{fsimage}} and {{edits}},
into image only directory  {{name}} and edits only {{editsDir}} I see that {{name}} still
contains the edits file. The same happens if I split out the image directory. The old one
will still contain {{fsimage}}.

Files that do not correspond to the current directory type should be removed be saveFSImage().

I initially thought it is not a bug. But in fact it is.
Suppose I have an image-only directory {{imageDir}}, which is used to be image+edits directory
and therefore contains an old file "edits". I also have an edits-only directory {{editsDir}},
which contains the up to date journal.
Then I by mistake configure  {{imageDir}} to be combined image+edits directory. Since {{fstime}}
is the same in {{imageDir}} and {{editsDir}} any of the two edits files can be chosen by the
name-node during startup. And if it chooses the old one from {{imageDir}} latest mutations
to the name space will be lost.

> saveFSImage() should remove files from a storage directory that do not correspond to
its type.
> ----------------------------------------------------------------------------------------------
>                 Key: HADOOP-4404
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4404
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.19.0
>            Reporter: Konstantin Shvachko
>            Priority: Blocker
>             Fix For: 0.19.0
> NameNode's image and edits directories can be separated (HADOOP-3948).
> So the name-node can have 3 types of storage directories. 
> And each directory should contain only those files that correspond to its type:
> # combined image and edits directory: should contain {{fsimage}} and {{edits}}
> # image only directory should contain only {{fsimage}}
> # edits only directory should contain only {{edits}}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message