Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 4826B200D4E for ; Thu, 7 Dec 2017 20:45:10 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 46B0D160C08; Thu, 7 Dec 2017 19:45:10 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8F14E160C0C for ; Thu, 7 Dec 2017 20:45:09 +0100 (CET) Received: (qmail 52315 invoked by uid 500); 7 Dec 2017 19:45:08 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 52303 invoked by uid 99); 7 Dec 2017 19:45:08 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Dec 2017 19:45:08 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 3EDE61A038D for ; Thu, 7 Dec 2017 19:45:08 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id EaHvUNcsPEfx for ; Thu, 7 Dec 2017 19:45:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 6A037629D4 for ; Thu, 7 Dec 2017 19:45:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id D4E46E0732 for ; Thu, 7 Dec 2017 19:45:00 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 8D44121E76 for ; Thu, 7 Dec 2017 19:45:00 +0000 (UTC) Date: Thu, 7 Dec 2017 19:45:00 +0000 (UTC) From: "Hudson (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-19357) Bucket cache no longer L2 for LRU cache MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 07 Dec 2017 19:45:10 -0000 [ https://issues.apache.org/jira/browse/HBASE-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282406#comment-16282406 ] Hudson commented on HBASE-19357: -------------------------------- FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4185 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4185/]) HBASE-19357 Bucket cache no longer L2 for LRU cache. (anoopsamjohn: rev ba4f9f834948e6f042e771ae5ee016610afe928c) * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheConfig.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestColumnFamilyDescriptorBuilder.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java * (edit) hbase-common/src/main/resources/hbase-default.xml * (edit) hbase-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java * (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestImmutableHColumnDescriptor.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ColumnFamilyDescriptor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/InclusiveCombinedBlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HeapMemoryManager.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptorBuilder.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java * (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java * (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/util/MemorySizeUtil.java * (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ColumnFamilyDescriptorBuilder.java > Bucket cache no longer L2 for LRU cache > --------------------------------------- > > Key: HBASE-19357 > URL: https://issues.apache.org/jira/browse/HBASE-19357 > Project: HBase > Issue Type: Sub-task > Reporter: Anoop Sam John > Assignee: Anoop Sam John > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-19357.patch, HBASE-19357.patch, HBASE-19357_V2.patch, HBASE-19357_V3.patch, HBASE-19357_V3.patch > > > When Bucket cache is used, by default we dont configure it as an L2 cache alone. The default setting is combined mode ON where the data blocks to Bucket cache and index/bloom blocks go to LRU cache. But there is a way to turn this off and make LRU as L1 and Bucket cache as a victim handler for L1. It will be just L2. > After the off heap read path optimization Bucket cache is no longer slower compared to L1. We have test results on data sizes from 12 GB. The Alibaba use case was also with 12 GB and they have observed a ~30% QPS improve over the LRU cache. > This issue is to remove the option for combined mode = false. So when Bucket cache is in use, data blocks will go to it only and LRU will get only index /meta/bloom blocks. Bucket cache will no longer be configured as a victim handler for LRU. > Note : WHen external cache is in use, there only the L1 L2 thing comes. LRU will be L1 and external cache act as its L2. That make full sense. -- This message was sent by Atlassian JIRA (v6.4.14#64029)