hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chang Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9289) check genStamp when complete file
Date Mon, 26 Oct 2015 16:22:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14974490#comment-14974490

Chang Li commented on HDFS-9289:

have met another case in our cluster
2015-10-23 04:38:08,544 [IPC Server handler 11 on 8020] INFO hdfs.StateChange: BLOCK* allocateBlock:
gz.temp._COPYING_. BP-1161836467- blk_1427767166_354062734{blockUCState=UNDER_CONSTRUCTION,
primaryNodeIndex=-1, replicas=[Replica
f9f84c2c:NORMAL:|RBW], ReplicaUnderConstruction[[DISK]DS-14a850d1-deb9-496b-b5ed-bb57010a8b56:NORMAL:|RBW]]}
2015-10-23 04:39:35,588 [IPC Server handler 5 on 8020] INFO namenode.FSNamesystem: updatePipeline(block=BP-1161836467-
166_354062734, newGenerationStamp=354080525, newLength=24505255, newNodes=[,], clientName=DFSClient_NONMAPREDUCE_12621589
2015-10-23 04:39:35,588 [IPC Server handler 5 on 8020] INFO namenode.FSNamesystem: updatePipeline(BP-1161836467-
4062734) successfully to BP-1161836467-
2015-10-23 04:39:35,595 [IPC Server handler 50 on 8020] INFO hdfs.StateChange: DIR* completeFile:
/projects/wcc/wcc1/data/2015/10/22/05/Content-9892.temp.gz.temp._COPYING_ is closed by DFSClient_NONMAPREDUCE_1262158981_1
This is also a complete file before a pipelineupdate.
jsp page shows three nodes which currently holds the replica of blk_1427767166. One of the
node is, which is the first node in old pipeline and dropped out in the updated
pipeline and the replica currently in that node has old gen stamp. The other two nodes are
later replicated after the first node in old pipeline sent in its block report. The two nodes
in the updated pipeline were marked as corrupted until the node sent in its block

> check genStamp when complete file
> ---------------------------------
>                 Key: HDFS-9289
>                 URL: https://issues.apache.org/jira/browse/HDFS-9289
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Chang Li
>            Assignee: Chang Li
>            Priority: Critical
>         Attachments: HDFS-9289.1.patch, HDFS-9289.2.patch
> we have seen a case of corrupt block which is caused by file complete after a pipelineUpdate,
but the file complete with the old block genStamp. This caused the replicas of two datanodes
in updated pipeline to be viewed as corrupte. Propose to check genstamp when commit block

This message was sent by Atlassian JIRA

View raw message