Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <365132193.1254942092345.JavaMail.jira@brutus>
Date: Wed, 7 Oct 2009 12:01:32 -0700 (PDT)
From: "Hairong Kuang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-679) Block receiver unexpectedly throws an
 IOException complaining mismatched checksum
In-Reply-To: <960149890.1254872551520.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763192#action_12763192 ] 

Hairong Kuang commented on HDFS-679:
------------------------------------

I'd like to propose a solution to the appending to a partial checksum chunk problem. The client reads in the partial chunk into its buffer before any write and they will be part of the packets sent to the datanode side before the chunk gets filled up. This solution maintains a good property: any packet sent to datanodes starts at a checksum chunk boundary. This would not increase complexity at the client side but would completely eliminate the handling of appending to partial chunk at the datanode side. 

Another optimization to a block receiver at the datanode is not to overwrite block file if a packet starts with data that the replica file already has. For the crc file, only the last 4 bytes are allowed to be overwritten.

> Block receiver unexpectedly throws an IOException complaining mismatched checksum
> ---------------------------------------------------------------------------------
>
>                 Key: HDFS-679
>                 URL: https://issues.apache.org/jira/browse/HDFS-679
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.21.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.21.0
>
>
> When I run TestClientProtocolForPipelineRecovery, I always see that the block receiver throws IOException complaining about mismatched checksum when receiving the last data packet. It turns out the checksum of last packet was unexpectedly set to be zero.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.