hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11331) [blockcache] lazy block decompression
Date Sat, 26 Jul 2014 07:41:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14075297#comment-14075297
] 

stack commented on HBASE-11331:
-------------------------------

Seeing this... trying to figure it in client:

{code}2014-07-26 00:38:46,924 DEBUG [IPC Client (1094864774) connection to c2021.halxg.cloudera.com/10.20.84.27:16020
from stack] ipc.RpcClient: IPC Client (1094864774) connection to c2021.halxg.cloudera.com/10.20.84.27:16020
from stack: got response header call_id: 1102 exception { exception_class_name: "java.io.IOException"
stack_trace: "java.io.IOException\n\tat org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2047)\n\tat
org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:98)\n\tat org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)\n\tat
org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)\n\tat java.lang.Thread.run(Thread.java:744)\nCaused
by: java.lang.ArrayIndexOutOfBoundsException\n\tat org.apache.hadoop.hbase.io.hfile.HFileBlock.unpack(HFileBlock.java:498)\n\tat
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.getCachedBlock(HFileReaderV2.java:270)\n\tat
org.apache.hadoop.hbase.io.hfile.HFileReaderV2.readBlock(HFileReaderV2.java:424)\n\tat org.apache.hadoop.hbase.io.hfile.HFileBlockIndex$BlockIndexReader.loadDataBlockWithScanInfo(HFileBlockIndex.java:260)\n\tat
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.seekTo(HFileReaderV2.java:644)\n\tat
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$AbstractScannerV2.reseekTo(HFileReaderV2.java:624)\n\tat
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseekAtOrAfter(StoreFileScanner.java:251)\n\tat
org.apache.hadoop.hbase.regionserver.StoreFileScanner.reseek(StoreFileScanner.java:176)\n\tat
org.apache.hadoop.hbase.regionserver.NonLazyKeyValueScanner.doRealSeek(NonLazyKeyValueScanner.java:55)\n\tat
org.apache.hadoop.hbase.regionserver.KeyValueHeap.generalizedSeek(KeyValueHeap.java:312)\n\tat
org.apache.hadoop.hbase.regionserver.KeyValueHeap.requestSeek(KeyValueHeap.java:268)\n\tat
org.apache.hadoop.hbase.regionserver.StoreScanner.reseek(StoreScanner.java:721)\n\tat org.apache.hadoop.hbase.regionserver.StoreScanner.seekAsDirection(StoreScanner.java:709)\n\tat
org.apache.hadoop.hbase.regionserver.StoreScanner.next(StoreScanner.java:559)\n\tat org.apache.hadoop.hbase.regionserver.KeyValueHeap.next(KeyValueHeap.java:139)\n\tat
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.populateResult(HRegion.java:4002)\n\tat
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextInternal(HRegion.java:4082)\n\tat
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.nextRaw(HRegion.java:3951)\n\tat
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3933)\n\tat
org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.next(HRegion.java:3920)\n\tat
org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4865)\n\tat org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4838)\n\tat
org.apache.hadoop.hbase.regionserver.RSRpcServices.get(RSRpcServices.java:1619)\n\tat org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29990)\n\tat
org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2013)\n\t... 4 more\n" do_not_retry:
false }, totalSize: 2735 bytes{code}

> [blockcache] lazy block decompression
> -------------------------------------
>
>                 Key: HBASE-11331
>                 URL: https://issues.apache.org/jira/browse/HBASE-11331
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>         Attachments: HBASE-11331.00.patch, HBASE-11331.01.patch, HBASE-11331LazyBlockDecompressperfcompare.pdf
>
>
> Maintaining data in its compressed form in the block cache will greatly increase our
effective blockcache size and should show a meaning improvement in cache hit rates in well
designed applications. The idea here is to lazily decompress/decrypt blocks when they're consumed,
rather than as soon as they're pulled off of disk.
> This is related to but less invasive than HBASE-8894.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message