hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8150) Make getFileChecksum fail for blocks under construction
Date Thu, 14 May 2015 12:21:09 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14543575#comment-14543575

Hudson commented on HDFS-8150:

SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #196 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/196/])
HDFS-8150. Make getFileChecksum fail for blocks under construction (Contributed by J.Andreina)
(vinayakumarb: rev def9136e0259e118e6fd7b656260765d28ac9ae6)
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotFileLength.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestGetFileChecksum.java

> Make getFileChecksum fail for blocks under construction
> -------------------------------------------------------
>                 Key: HDFS-8150
>                 URL: https://issues.apache.org/jira/browse/HDFS-8150
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Kihwal Lee
>            Assignee: J.Andreina
>            Priority: Critical
>             Fix For: 2.8.0
>         Attachments: HDFS-8150.1.patch, HDFS-8150.2.patch, HDFS-8150.3.patch
> We have seen the cases of validating data copy using checksum then the content of target
changing. It turns out the target wasn't closed successfully, so it was still under-construction.
 One hour later, a lease recovery kicked in and truncated the block.
> Although this can be prevented in many ways, if there is no valid use case for getting
file checksum from under-construction blocks, can it be disabled?  E.g. Datanode can throw
an exception if the replica is not in the finalized state.

This message was sent by Atlassian JIRA

View raw message