From dev-return-68355-archive-asf-public=cust-asf.ponee.io@hbase.apache.org Fri Mar 2 02:20:06 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 74AB418064D for ; Fri, 2 Mar 2018 02:20:05 +0100 (CET) Received: (qmail 94517 invoked by uid 500); 2 Mar 2018 01:20:04 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 94486 invoked by uid 99); 2 Mar 2018 01:20:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Mar 2018 01:20:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 6D2FE180663 for ; Fri, 2 Mar 2018 01:20:03 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -110.311 X-Spam-Level: X-Spam-Status: No, score=-110.311 tagged_above=-999 required=6.31 tests=[ENV_AND_HDR_SPF_MATCH=-0.5, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, USER_IN_DEF_SPF_WL=-7.5, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id ayCoSn1CAn4c for ; Fri, 2 Mar 2018 01:20:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 973025F180 for ; Fri, 2 Mar 2018 01:20:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B99BAE0218 for ; Fri, 2 Mar 2018 01:20:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 6DF8524768 for ; Fri, 2 Mar 2018 01:20:00 +0000 (UTC) Date: Fri, 2 Mar 2018 01:20:00 +0000 (UTC) From: "stack (JIRA)" To: dev@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (HBASE-10403) Simplify offheap cache configuration MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack resolved HBASE-10403. --------------------------- Resolution: Fixed Resolving. Subtasks done. More to do here but enough for this JIRA > Simplify offheap cache configuration > ------------------------------------ > > Key: HBASE-10403 > URL: https://issues.apache.org/jira/browse/HBASE-10403 > Project: HBase > Issue Type: Improvement > Components: io > Reporter: Nick Dimiduk > Assignee: Nick Dimiduk > Priority: Minor > Fix For: 2.0.0 > > Attachments: HBASE-10403.0.patch > > > The BucketCache (HBASE-7404) is a very nice piece of functionality which is hidden behind complex configuration. Enabling it currently requires manual calculation of L1 cache. It'd be nice to make this easier to use and conform better with the existing heap management tools we already have. > Turning it on currently requires explicitly setting LruBlockCache (L1) instance size and IOEngine (L2) size, making sure that L1 size isn't too big vs global memstore and total heap. This is further confused by hbase.bucketcache.size accepting a percentage of total heap OR explicit size in MB. Enabling SlabCache is slightly easier in that it just accepts whatever LruBlockCache is provided. > Turning on BucketCache using off-heap mode could look like: > hbase-env.sh: > Set HBASE_REGIONSERVER_OPTS: > -Xmx5000m > -XX:MaxDirectMemorySize=15000m > hbase-site.xml: > - hbase.regionserver.global.memstore.size = 0.7 > - hbase.regionserver.onheap.blockcache.size = 0.1 > - hbase.regionserver.blockcache.impl = BucketCache > - hbase.bucketcache.ioengine = offheap > The result being a CombinedCache instance with 500m LruBlockCache + 15000m ByteBufferIOEngine running in direct mode. > This example does a couple things (mostly for the admin): > - knows NOT to enable SlabCache > - s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/ > - maintains the validity of HBaseConfiguration's existing check that global MemStore + LruBlockCache == 0.8 > - maps "BucketCache" into meaning "a CombinedCache instance with these implementations for L1 and L2." > - Figures out appropriate values for hbase.bucketcache.size and hbase.bucketcache.percentage.in.combinedcache -- This message was sent by Atlassian JIRA (v7.6.3#76005)