hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9289) Make DataStreamer#block thread safe and verify genStamp in commitBlock
Date Mon, 02 Nov 2015 23:50:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14986303#comment-14986303
] 

Zhe Zhang commented on HDFS-9289:
---------------------------------

The test looks reasonable. I can see 2 minor issues:
# {{DFSTestUtil#addBlockToFile}} is very similar as {{addStripedBlockToFile}}. At the minimum
we should change the comment (not adding a "block group"). Ideally we should consolidate the
two methods with a switch for striped or not.
# The {{TestCommitBlock}} class only has 1 test and should be named more specifically, something
like {{TestCommitBlockWithInvalidGenStamp}}

If we create a new {{InvalidGenStampException}} class, I guess we should apply it to all GS-related
NN exceptions? E.g. {{BlockManager#checkReplicaCorrupt}}. How about we do it under a separate
JIRA?

> Make DataStreamer#block thread safe and verify genStamp in commitBlock
> ----------------------------------------------------------------------
>
>                 Key: HDFS-9289
>                 URL: https://issues.apache.org/jira/browse/HDFS-9289
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Chang Li
>            Assignee: Chang Li
>            Priority: Critical
>         Attachments: HDFS-9289.1.patch, HDFS-9289.2.patch, HDFS-9289.3.patch, HDFS-9289.4.patch,
HDFS-9289.5.patch, HDFS-9289.6.patch
>
>
> we have seen a case of corrupt block which is caused by file complete after a pipelineUpdate,
but the file complete with the old block genStamp. This caused the replicas of two datanodes
in updated pipeline to be viewed as corrupte. Propose to check genstamp when commit block



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message