hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11625) Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum
Date Tue, 03 May 2016 22:57:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269794#comment-15269794
] 

Appy commented on HBASE-11625:
------------------------------

Here's how to reproduce.
I have uploaded two files:
- correct-hfile : Generated using hbase shell to write two rows to a new table and flushing
it.
- corrupted-header-hfile: Changed one character in 'correct-hfile' belonging to block magic.
Use {{cmp -b correct-hfile corrupted-header-hfile}} to see the diff.
Since I messed with the file, it should report that data was corrupted but instead it throws
'Invalid hfile block magic' exception (see below output).
The fix, as mentioned above, is to first do checksum and then make HFileBlock instance.

{noformat}
appy@asharma-MBP ~/apache/hbase/bin  (master) → ./hbase org.apache.hadoop.hbase.io.hfile.HFile
-p -f file:///Users/appy/apache/hbase/correct-hfile
2016-05-03 15:47:37,332 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
2016-05-03 15:47:37,534 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
K: 1/cf:1/1462311794785/Put/vlen=5/seqid=4 V: 11111
K: 1/cf:2/1462311803576/Put/vlen=6/seqid=5 V: 222222
Scanned kv count -> 2

appy@asharma-MBP ~/apache/hbase/bin  (master) → ./hbase org.apache.hadoop.hbase.io.hfile.HFile
-p -f file:///Users/appy/apache/hbase/corrupted-header-hfile
2016-05-03 15:47:43,932 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
2016-05-03 15:47:44,089 INFO  [main] hfile.CacheConfig: CacheConfig:disabled
2016-05-03 15:47:44,301 ERROR [main] hfile.HFilePrettyPrinter: Error reading file:/Users/appy/apache/hbase/corrupted-header-hfile
java.io.IOException: Invalid HFile block magic: DcTABLK*
	at org.apache.hadoop.hbase.io.hfile.BlockType.parse(BlockType.java:154)
	at org.apache.hadoop.hbase.io.hfile.BlockType.read(BlockType.java:167)
	at org.apache.hadoop.hbase.io.hfile.HFileBlock.<init>(HFileBlock.java:255)
	at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockDataInternal(HFileBlock.java:1684)
	at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderImpl.readBlockData(HFileBlock.java:1507)
	at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:446)
	at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:914)
	at org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.processFile(HFilePrettyPrinter.java:269)
	at org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.run(HFilePrettyPrinter.java:230)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.main(HFilePrettyPrinter.java:552)
	at org.apache.hadoop.hbase.io.hfile.HFile.main(HFile.java:879)
{noformat}



> Reading datablock throws "Invalid HFile block magic" and can not switch to hdfs checksum

> -----------------------------------------------------------------------------------------
>
>                 Key: HBASE-11625
>                 URL: https://issues.apache.org/jira/browse/HBASE-11625
>             Project: HBase
>          Issue Type: Bug
>          Components: HFile
>    Affects Versions: 0.94.21, 0.98.4, 0.98.5, 1.0.1.1, 1.0.3
>            Reporter: qian wang
>            Assignee: Pankaj Kumar
>             Fix For: 2.0.0
>
>         Attachments: 2711de1fdf73419d9f8afc6a8b86ce64.gz, HBASE-11625.patch, correct-hfile,
corrupted-header-hfile
>
>
> when using hbase checksum,call readBlockDataInternal() in hfileblock.java, it could happen
file corruption but it only can switch to hdfs checksum inputstream till validateBlockChecksum().
If the datablock's header corrupted when b = new HFileBlock(),it throws the exception "Invalid
HFile block magic" and the rpc call fail



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message