Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 36BE91093E for ; Wed, 31 Dec 2014 02:34:13 +0000 (UTC) Received: (qmail 3085 invoked by uid 500); 31 Dec 2014 02:34:13 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 3039 invoked by uid 500); 31 Dec 2014 02:34:13 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 3027 invoked by uid 99); 31 Dec 2014 02:34:13 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 31 Dec 2014 02:34:13 +0000 Date: Wed, 31 Dec 2014 02:34:13 +0000 (UTC) From: "Liu Shaohui (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-12270) A bug in the bucket cache, with cache blocks on write enabled MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-12270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261815#comment-14261815 ] Liu Shaohui commented on HBASE-12270: ------------------------------------- Sorry for the patch make the build of trunk and 0.98 failed. I will fix the TestCacheOnWrite today. > A bug in the bucket cache, with cache blocks on write enabled > ------------------------------------------------------------- > > Key: HBASE-12270 > URL: https://issues.apache.org/jira/browse/HBASE-12270 > Project: HBase > Issue Type: Bug > Affects Versions: 0.94.11, 0.98.6.1 > Environment: I can reproduce it on a simple 2 node cluster, one running the master and another running a RS. I was testing on ec2. > I used the following configurations for the cluster. > hbase-env:HBASE_REGIONSERVER_OPTS=-Xmx2G -XX:MaxDirectMemorySize=5G -XX:CMSInitiatingOccupancyFraction=88 -XX:+AggressiveOpts -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -Xlog gc:/tmp/hbase-regionserver-gc.log > hbase-site: > hbase.bucketcache.ioengine=offheap > hbase.bucketcache.size=4196 > hbase.rs.cacheblocksonwrite=true > hfile.block.index.cacheonwrite=true > hfile.block.bloom.cacheonwrite=true > Reporter: Khaled Elmeleegy > Assignee: Liu Shaohui > Priority: Critical > Fix For: 1.0.0, 2.0.0, 0.98.10, 1.1.0 > > Attachments: HBASE-12270-v1.diff, HBASE-12270-v2.diff, HBASE-12270-v2.patch, TestHBase.java, TestKey.java > > > In my experiments, I have writers streaming their output to HBase. The reader powers a web page and does this scatter/gather, where it reads 1000 keys written last and passes them the the front end. With this workload, I get the exception below at the region server. Again, I am using HBAse (0.98.6.1). Any help is appreciated. > 2014-10-10 15:06:44,173 ERROR [B.DefaultRpcServer.handler=62,queue=2,port=60020] ipc.RpcServer: Unexpected throwable object > java.lang.IllegalArgumentException > at java.nio.Buffer.position(Buffer.java:236) > at org.apache.hadoop.hbase.util.ByteBufferUtils.skip(ByteBufferUtils.java:434) > at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readKeyValueLen(HFileReaderV2.java:849) > at org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.next(HFileReaderV2.java:760) > at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:248) > at org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:152) > at org.apache.hadoop.hbase.regionserver.StoreScanner.seekScanners(StoreScanner.java:317) > at org.apache.hadoop.hbase.regionserver.StoreScanner.(StoreScanner.java:176) > at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1780) > at org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.(HRegion.java:3758) > at org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1950) > at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1936) > at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1913) > at org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3157) > at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29587) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114) > at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94) > at java.lang.Thread.run(Thread.java:744) -- This message was sent by Atlassian JIRA (v6.3.4#6332)