hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shankar hiremath <shankar.hirem...@huawei.com>
Subject Problem reading HFile Trailer by using HFile tool
Date Fri, 01 Aug 2014 14:12:35 GMT
When I am reading the Hfile using the './hbase org.apache.hadoop.hbase.io.hfile.Hfile' tool
(the HFile which is compressed with SNAPPY & encrypted with AES)
I am getting the below error "Problem reading HFile Trailer by using HFile tool"

Is there any problem in the usage of the below command, or is it a bug.

------------------------------------------------------------------------------------------

Version details: Hadoop 2.4.1, HBase 0.98.3

Configuration Details: (enabled Hfile and Wal encryption AES, as below)
hfile.format.version=3
hbase.crypto.keyprovider=org.apache.hadoop.hbase.io.crypto.KeyStoreKeyProvider
hbase.crypto.keyprovider.parameters= jceks:///opt/shankar1/kdc_keytab/hbase.jks?password=shankar@234

We created a table with SNAPPY  compression
>       Create 't3', {NAME => 'cf1', COMPRESSION => 'SNAPPY'}
>       Put 't3','r1','cf1:a','1000'
>       Flush 't3'

shankar1@host1:~/DataSight/hbase/bin> ./hbase org.apache.hadoop.hbase.io.hfile.HFile -v
-f hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
2014-08-01 19:18:28,368 INFO  [main] Configuration.deprecation: hadoop.native.lib is deprecated.
Instead, use io.native.lib.available
2014-08-01 19:18:28,504 INFO  [main] util.ChecksumType: Checksum using org.apache.hadoop.util.PureJavaCrc32
2014-08-01 19:18:28,506 INFO  [main] util.ChecksumType: Checksum can use org.apache.hadoop.util.PureJavaCrc32C
2014-08-01 19:18:28,739 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
2014-08-01 19:18:29,082 INFO  [main] hdfs.DFSClient: Set dfs.client.block.write.replace-datanode-on-failure.replication
to 0
2014-08-01 19:18:29,406 INFO  [main] Configuration.deprecation: fs.default.name is deprecated.
Instead, use fs.defaultFS
Scanning -> hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
2014-08-01 19:18:29,409 INFO  [main] hdfs.DFSClient: Set dfs.client.block.write.replace-datanode-on-failure.replication
to 0
INFO: Watching file:/opt/shankar1/DataSight/hbase/conf/log4j.properties for changes with interval
: 60000
2014-08-01 19:18:29,779 ERROR [main] hfile.HFilePrettyPrinter: Error reading hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
org.apache.hadoop.hbase.io.hfile.CorruptHFileException: Problem reading HFile Trailer from
file hdfs://host1:65110/hbase/data/default/t3/337d2996bed579340a702feaa3d3f165/cf1/5817635667d7457989b6d0b0be25dbc4
        at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:552)
        at org.apache.hadoop.hbase.io.hfile.HFile.createReader(HFile.java:595)
        at org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.processFile(HFilePrettyPrinter.java:217)
        at org.apache.hadoop.hbase.io.hfile.HFilePrettyPrinter.run(HFilePrettyPrinter.java:196)
        at org.apache.hadoop.hbase.io.hfile.HFile.main(HFile.java:873)
Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
        at org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
        at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
        at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:190)
        at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:176)
        at org.apache.hadoop.hbase.io.compress.Compression$Algorithm.getDecompressor(Compression.java:336)
        at org.apache.hadoop.hbase.io.compress.Compression.decompress(Compression.java:433)
        at org.apache.hadoop.hbase.io.encoding.HFileBlockDefaultDecodingContext.prepareDecoding(HFileBlockDefaultDecodingContext.java:91)
        at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockDataInternal(HFileBlock.java:1522)
        at org.apache.hadoop.hbase.io.hfile.HFileBlock$FSReaderV2.readBlockData(HFileBlock.java:1314)
        at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlock(HFileBlock.java:1151)
        at org.apache.hadoop.hbase.io.hfile.HFileBlock$AbstractFSReader$1.nextBlockWithBlockType(HFileBlock.java:1159)
        at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.<init>(HFileReaderV2.java:146)
        at org.apache.hadoop.hbase.io.hfile.HFileReaderV3.<init>(HFileReaderV3.java:72)
        at org.apache.hadoop.hbase.io.hfile.HFile.pickReaderVersion(HFile.java:542)
        ... 4 more
Scanned kv count -> 0
shankar1@host1:~/DataSight/hbase/bin>

Thanks
-Shankar

[X]
This e-mail and its attachments contain confidential information from HUAWEI, which is intended
only for the person or entity whose address is listed above. Any use of the information contained
herein in any way (including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive
this e-mail in error, please notify the sender by phone or email immediately and delete it!
[X]






Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message