hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14046) IllegalArgumentException: Negative position from hfile.bucket.FileIOEngine.read
Date Wed, 04 Nov 2015 05:55:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988944#comment-14988944
] 

ramkrishna.s.vasudevan commented on HBASE-14046:
------------------------------------------------

Though not able to find the root cause - in the case I got this bug it was like all my data
were major compacted. One of the Hfile root index offset either should be corrupted while
writing or may be there is an issue in some specific case while reading and filling up the
index offsets. But this bug is surely lurking some where in the code. Was just analysing the
code to see if there could be a int overflow or something to happen and that we try to use
in the read path, seems that is clear. 

> IllegalArgumentException: Negative position from hfile.bucket.FileIOEngine.read
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-14046
>                 URL: https://issues.apache.org/jira/browse/HBASE-14046
>             Project: HBase
>          Issue Type: Bug
>          Components: hbase
>    Affects Versions: 0.98.6
>         Environment: CentOS release 6.4 X86_64
>            Reporter: louis hust
>            Priority: Critical
>
> When I scan a table using hbase shell, got the following message:
> {code:bash}
> hbase(main):001:0> scan 'atpco:ttf_record6'
> ROW                                              COLUMN+CELL
> ERROR: org.apache.hadoop.hbase.exceptions.OutOfOrderScannerNextException: Expected nextCallSeq:
1 But the nextCallSeq got from client: 0; request=scanner_id: 201542113 number_of_rows: 100
close_scanner: false next_call_seq: 0
> 	at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3193)
> 	at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29587)
> 	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
> 	at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> 	at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
> 	at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> 	at java.lang.Thread.run(Thread.java:744)
> {code}
> And the region server got the following error:
> {code}
> 2015-06-30 11:08:11,877 ERROR [B.defaultRpcServer.handler=27,queue=0,port=60020] ipc.RpcServer:
Unexpected throwable object
> java.lang.IllegalArgumentException: Negative position
>         at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:675)
>         at org.apache.hadoop.hbase.io.hfile.bucket.FileIOEngine.read(FileIOEngine.java:87)
>         at org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getBlock(BucketCache.java:406)
>         at org.apache.hadoop.hbase.io.hfile.LruBlockCache.getBlock(LruBlockCache.java:389)
>         at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:359)
>         at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.readNextDataBlock(HFileReaderV2.java:635)
>         at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:749)
>         at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:136)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:108)
>         at org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:507)
>         at org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:140)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:3900)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:3980)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3858)
>         at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3849)
>         at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3245)
>         at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29587)
>         at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
>         at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> {code}
> It seems something wrong with the BucketCache.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message