hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1481) NameNode should validate fsimage before rolling
Date Thu, 11 Nov 2010 08:41:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12930963#action_12930963
] 

Konstantin Shvachko commented on HDFS-1481:
-------------------------------------------

The patch looks good.
It verifies the checksum, when NN uploads the checkpoint image.
Looking at the code it seems that we can also verify the checksum when CN loads the image
from NN, when the checkpoint starts. It seems as simple as calling TransferFsImage.getFileClient()
with getChecksum = true instead of false in downloadCheckpoint(), since the signature already
has imageDigest.
For SNN it doesn't look as simple.

> NameNode should validate fsimage before rolling
> -----------------------------------------------
>
>                 Key: HDFS-1481
>                 URL: https://issues.apache.org/jira/browse/HDFS-1481
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.22.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.22.0
>
>         Attachments: trunkValidateUpload.patch
>
>
> We had an incident that the fsimage at secondary NameNode was truncated but got uploaded
to the primary NameNode. The primary NameNode simply rolled the image without checking its
integrity, therefore causing the fsimage to corrupt. The primary NameNode should check the
new image's integrity before rolling fsimage.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message