hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liang Xie (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8894) Forward port compressed l2 cache from 0.89fb
Date Fri, 18 Oct 2013 10:52:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13798998#comment-13798998
] 

Liang Xie commented on HBASE-8894:
----------------------------------

I had run an apple to apple perf test this afternoon, shows this l2 cache is worse than orignial
bucket cache.  In my 100% read only scenario, with same Xmx/Directmemory/(l2).bucketcache
size setting, the response time and  us/sys cpu% were very stable in bucket cache, but very
unstable on l2 cache, more worse things is in my scenario, there's 0 fgc for bucket cache,
but lots of fgc be observed for l2 cache,  my explaination is the data block could goto heap's
lru block cache, that makes heap fragment happened again,  and then, the fgc's cost beat all
the earnings of l2 cache (like get rid of deserialization)

> Forward port compressed l2 cache from 0.89fb
> --------------------------------------------
>
>                 Key: HBASE-8894
>                 URL: https://issues.apache.org/jira/browse/HBASE-8894
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: stack
>            Assignee: Liang Xie
>            Priority: Critical
>         Attachments: HBASE-8894-0.94-v1.txt, HBASE-8894-0.94-v2.txt
>
>
> Forward port Alex's improvement on hbase-7407 from 0.89-fb branch:
> {code}
>   1 r1492797 | liyin | 2013-06-13 11:18:20 -0700 (Thu, 13 Jun 2013) | 43 lines
>   2
>   3 [master] Implements a secondary compressed cache (L2 cache)
>   4
>   5 Author: avf
>   6
>   7 Summary:
>   8 This revision implements compressed and encoded second-level cache with off-heap
>   9 (and optionally on-heap) storage and a bucket-allocator based on HBASE-7404.
>  10
>  11 BucketCache from HBASE-7404 is extensively modified to:
>  12
>  13 * Only handle byte arrays (i.e., no more serialization/deserialization within)
>  14 * Remove persistence support for the time being
>  15 * Keep an  index of hfilename to blocks for efficient eviction on close
>  16
>  17 A new interface (L2Cache) is introduced in order to separate it from the current
>  18 implementation. The L2 cache is then integrated into the classes that handle
>  19 reading from and writing to HFiles to allow cache-on-write as well as
>  20 cache-on-read. Metrics for the L2 cache are integrated into RegionServerMetrics
>  21 much in the same fashion as metrics for the existing (L2) BlockCache.
>  22
>  23 Additionally, CacheConfig class is re-refactored to configure the L2 cache,
>  24 replace multile constructors with a Builder, as well as replace static methods
>  25 for instantiating the caches with abstract factories (with singleton
>  26 implementations for both the existing LruBlockCache and the newly introduced
>  27 BucketCache based L2 cache)
>  28
>  29 Test Plan:
>  30 1) Additional unit tests
>  31 2) Stress test on a single devserver
>  32 3) Test on a single-node in shadow cluster
>  33 4) Test on a whole shadow cluster
>  34
>  35 Revert Plan:
>  36
>  37 Reviewers: liyintang, aaiyer, rshroff, manukranthk, adela
>  38
>  39 Reviewed By: liyintang
>  40
>  41 CC: gqchen, hbase-eng@
>  42
>  43 Differential Revision: https://phabricator.fb.com/D837264
>  44
>  45 Task ID: 2325295
>  7 ------------------------------------------------------------------------
>   6 r1492340 | liyin | 2013-06-12 11:36:03 -0700 (Wed, 12 Jun 2013) | 21 lines
> {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message