hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yu Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17747) Support both weak and soft object pool
Date Tue, 14 Mar 2017 08:24:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15923774#comment-15923774

Yu Li commented on HBASE-17747:

bq. And for the YCSB test, does it mean that all the data could be cached in blockcache? If
so then I think strong reference will perform best...
Correct, but we cannot use strong reference or else we need to clear the lock object by ourselves,
all the way back to the old style and read-write lock will be impossible. See more discussions
about this in HBASE-14463...

bq. And out of interest, why you have such a large heap size compare to the off heap size?
For write-heavy workload?
Simply using the -Xmx from online configuration, 40g*0.3=12g, equal to offheap bucket cache.

bq. I assume that 'dirty card table' is some algorithm for GC...
Aye, both CMS and G1 use card table, but it's truly implementation stuff, not something in
JVM specification...

bq. You need to convince others to accept your patch before commit, not commit it first then
let others convince you to revert it.
I just feel hard to convince you (for CMS GC) if you don't see it personally (smile). And
for this case, I don't think anyone to blame, it's common that ignore something during code
review, but I got your +1 in RB and wait one day for objections before commit it... And now
that it's already in but you found out some new points, I think discussions even arguments
are necessary until we get a good reason to let it be or revert it, right?...

So, to confirm, that we seem to get a consensus on using soft reference is OK in theory, but
better to have: 1) more testing with G1 GC; and 2) more online observation with CMS GC, before
final decision. And since the commit is already in (no one to blame, I insist (smile) ), with
quite some testing although maybe not enough, we use HBASE-17782 to make it configurable and
change the default if found any issue with soft reference later. Deal? [~Apache9]

> Support both weak and soft object pool
> --------------------------------------
>                 Key: HBASE-17747
>                 URL: https://issues.apache.org/jira/browse/HBASE-17747
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 2.0
>            Reporter: Yu Li
>            Assignee: Yu Li
>             Fix For: 2.0
>         Attachments: HBASE-17747.patch, HBASE-17747.v2.patch, HBASE-17747.v3.patch
> During YCSB testing on embedded mode after HBASE-17744, we found that under high read
load GC is quite severe even with offheap L2 cache. After some investigation, we found it's
caused by using weak reference in {{IdReadWriteLock}}. In embedded mode the read is so quick
that the lock might already get promoted to the old generation when the weak reference is
cleared, which causes dirty card table (old reference get removed and new lock object set
into {{referenceCache}}, see {{WeakObjectPool#get}}) thus slowing YGC. In distributed mode
there'll also be more lock object created with weak reference than soft reference that slowing
down the processing.
> So we proposed to use soft reference for this {{IdReadWriteLock}} used in cache, which
won't get cleared until JVM memory is not enough, and could resolve the issue mentioned above.
What's more, we propose to extend the {{WeakObjectPool}} to be more generate to support both
weak and soft reference.
> Note that the GC issue only emerges under embedded mode with DirectOperator, in which
case all costs on the wire is removed thus produces extremely high concurrency.

This message was sent by Atlassian JIRA

View raw message