hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19357) Bucket cache no longer L2 for LRU cache
Date Thu, 07 Dec 2017 19:45:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282406#comment-16282406
] 

Hudson commented on HBASE-19357:
--------------------------------

FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4185 (See [https://builds.apache.org/job/HBase-Trunk_matrix/4185/])
HBASE-19357 Bucket cache no longer L2 for LRU cache. (anoopsamjohn: rev ba4f9f834948e6f042e771ae5ee016610afe928c)
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheConfig.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/AccessController.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerWrapperImpl.java
* (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestColumnFamilyDescriptorBuilder.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestHeapMemoryManager.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
* (edit) hbase-common/src/main/resources/hbase-default.xml
* (edit) hbase-external-blockcache/src/main/java/org/apache/hadoop/hbase/io/hfile/MemcachedBlockCache.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestCacheOnWrite.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/bucket/TestBucketCache.java
* (edit) hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestImmutableHColumnDescriptor.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ColumnFamilyDescriptor.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/InclusiveCombinedBlockCache.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HeapMemoryManager.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/HColumnDescriptor.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/HFileReaderImpl.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptorBuilder.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CacheConfig.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/CombinedBlockCache.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSTableDescriptors.java
* (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/io/util/MemorySizeUtil.java
* (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/client/ColumnFamilyDescriptorBuilder.java


> Bucket cache no longer L2 for LRU cache
> ---------------------------------------
>
>                 Key: HBASE-19357
>                 URL: https://issues.apache.org/jira/browse/HBASE-19357
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0-beta-1
>
>         Attachments: HBASE-19357.patch, HBASE-19357.patch, HBASE-19357_V2.patch, HBASE-19357_V3.patch,
HBASE-19357_V3.patch
>
>
> When Bucket cache is used, by default we dont configure it as an L2 cache alone. The
default setting is combined mode ON where the data blocks to Bucket cache and index/bloom
blocks go to LRU cache. But there is a way to turn this off and make LRU as L1 and Bucket
cache as a victim handler for L1. It will be just L2.   
> After the off heap read path optimization Bucket cache is no longer slower compared to
L1. We have test results on data sizes from 12 GB.  The Alibaba use case was also with 12
GB and they have observed a ~30% QPS improve over the LRU cache.
> This issue is to remove the option for combined mode = false. So when Bucket cache is
in use, data blocks will go to it only and LRU will get only index /meta/bloom blocks.   Bucket
cache will no longer be configured as a victim handler for LRU.
> Note : WHen external cache is in use, there only the L1 L2 thing comes. LRU will be L1
and external cache act as its L2. That make full sense.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message