hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-957) FSImage layout version should be only once file is complete
Date Wed, 10 Feb 2010 01:34:28 GMT

    [ https://issues.apache.org/jira/browse/HDFS-957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831803#action_12831803
] 

Konstantin Shvachko commented on HDFS-957:
------------------------------------------

# I don't understand what problem this solves. We always save new image into IMAGE_NEW. Then
we rename it to IMAGE. If IMAGE_NEW exists we treat it as if the save did not complete (unsuccessful
checkpoint or save). IMAGE_NEW then is simply removed and the old image in IMAGE file is used
as the current image. There is no need to track progress of writing into IMAGE_NEW.
# It is very important in upgrade that we do not write anything into VERSION file until IMAGE
file is written. Writing VERSION file is like a commit when the image directory is formed.
We first write everything else then VERSION file, which indicates the directory is in good
order.
If you create VERSION file before completing the image all this logic will not work.

> FSImage layout version should be only once file is complete
> -----------------------------------------------------------
>
>                 Key: HDFS-957
>                 URL: https://issues.apache.org/jira/browse/HDFS-957
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.22.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>         Attachments: hdfs-957.txt
>
>
> Right now, the FSImage save code writes the LAYOUT_VERSION at the head of the file, along
with some other headers, and then dumps the directory into the file. Instead, it should write
a special IMAGE_IN_PROGRESS entry for the layout version, dump all of the data, then seek
back to the head of the file to write the proper LAYOUT_VERSION. This would make it very easy
to detect the case where the FSImage save got interrupted.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message