hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abhishek Kumar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18030) Negative tag length for hbase block/hfile
Date Thu, 11 May 2017 06:49:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16005974#comment-16005974

Abhishek Kumar commented on HBASE-18030:

My table is having two column family (weekly, daily) and using TTL.
For me, these are very basic features for tables with increments. The weekly column family
has small files and I don't see them compacting (total 5 HFiles). This is not the case with
daily column family, which has just 1 HFile.

hbase(main):016:0> get 'table-name', 'rowkey', 'daily:a'
0 row(s) in 0.0100 seconds

hbase(main):017:0> get 'table-name', 'rowkey', 'weekly:a'
ERROR: java.io.IOException: Invalid currTagsLen -32723. Block offset: 0, block length: 69222,
position: 3447 (without header).
	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2172)

A cron runs at my application service (3 of them) which flushes all the increments it accumulated,
one by one to hbase.

Nothing suspicious about the whole flow or any patterns I have observed. 
I am trying to remove TTL and moving to just 1 column family to see if it is fixing the issue.

> Negative tag length for hbase block/hfile
> -----------------------------------------
>                 Key: HBASE-18030
>                 URL: https://issues.apache.org/jira/browse/HBASE-18030
>             Project: HBase
>          Issue Type: Bug
>          Components: hbase, regionserver
>    Affects Versions: 1.1.2
>            Reporter: Abhishek Kumar
> 2017-04-29 14:24:14,135 ERROR [B.fifo.QRpcServer.handler=49,queue=1,port=16020] ipc.RpcServer:
Unexpected throwable object java.lang.IllegalStateException: Invalid currTagsLen -32712. Block
offset: 3707853, block length: 72841, position: 0 (without header). at org.apache.hadoop.hbase.io.hfile.HFileReaderV3$ScannerV3.checkTagsLen(HFileReaderV3.java:226)
> I am not not using any hbase tags feature.
> The Increment operation from the application side is triggering this error. The same
is happening when scanner is run on this table. It feels that one or more particular HFile
block is corrupt (with negative tagLength).
> hbase(main):007:0> scan 'table-name', {LIMIT=>1,STARTROW=>'ad:event_count:a'}
> Returning the result
> hbase(main):008:0> scan 'table-name', {LIMIT=>1,STARTROW=>'ad:event_count:b'}
> ROW                                                         COLUMN+CELL             
> ERROR: java.io.IOException: java.lang.IllegalStateException: Invalid currTagsLen -32701.
Block offset: 272031, block length: 72441, position: 32487 (without header).
> 	at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.handleException(HRegion.java:5607)
> 	at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:5579)
> 	at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:2627)
> 	at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2613)
> 	at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:2595)
> 	at org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2282)
> 	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32295)

This message was sent by Atlassian JIRA

View raw message