hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
Date Wed, 27 Apr 2016 19:50:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260814#comment-15260814

Appy commented on HBASE-11625:

Ahhh, the reason for the bug seems quite simple and has always been there in the description.
I guess we all missed it because it's written in a bad way. 
Breaking it down.
We check for block magic inside [new HFileBlock(), line 1697|https://github.com/apache/hbase/blob/12f66e3060339acc569ee425385e2dc255bb3e94/hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java#L1697]
(see the below stack trace)  and we verify the checksum after that on line 1705. That's using
the data before verifying that it's correct. So if the data is actually corrupted, we wrongly
throw the exception 'invalid hfile block magic', whereas we should have fallen back to hdfs
for getting correct data.

> Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum

> -----------------------------------------------------------------------------------------
>                 Key: HBASE-11625
>                 URL: https://issues.apache.org/jira/browse/HBASE-11625
>             Project: HBase
>          Issue Type: Bug
>          Components: HFile
>    Affects Versions: 0.94.21, 0.98.4, 0.98.5,, 1.0.3
>            Reporter: qian wang
>            Assignee: Pankaj Kumar
>             Fix For: 2.0.0
>         Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, HBASE-11625.patch
> when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it could happen
file corruption but it only can switch to hdfs checksum inputstream till validateBlockChecksum().
If the datablock's header corrupted when b = new HFileBlock(),it throws the exception "Invalid
HFile block magic" and the rpc call fail

This message was sent by Atlassian JIRA

View raw message