hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kousuke Saruta (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-5761) DataNode fail to validate integrity for checksum type NULL when DataNode recovers
Date Mon, 13 Jan 2014 16:06:51 GMT
Kousuke Saruta created HDFS-5761:
------------------------------------

             Summary: DataNode fail to validate integrity for checksum type NULL when DataNode
recovers 
                 Key: HDFS-5761
                 URL: https://issues.apache.org/jira/browse/HDFS-5761
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: datanode
    Affects Versions: 3.0.0
            Reporter: Kousuke Saruta
            Assignee: Kousuke Saruta


When DataNode is down during writing blocks, the blocks are not filinalized and the next time
DataNode recovers, integrity validation will run.
But if we use NULL for checksum algorithm (we can set NULL to dfs.checksum.type), DataNode
will fail to validate integrity and cannot be up. 

The cause is in BlockPoolSlice#validateIntegrity.
In the method, there is following code.

{code}
long numChunks = Math.min(
          (blockFileLen + bytesPerChecksum - 1)/bytesPerChecksum, 
          (metaFileLen - crcHeaderLen)/checksumSize);
{code}

When we choose NULL checksum, checksumSize is 0 so ArithmeticException will be thrown and
DataNode cannot be up.




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message