hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-12856) Revisit table coprocessor classloader caching
Date Wed, 14 Jan 2015 19:05:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277454#comment-14277454
] 

Lars Hofhansl edited comment on HBASE-12856 at 1/14/15 7:05 PM:
----------------------------------------------------------------

+1

Weak refs are wrong anyway, they are only useful to access something that is strongly referenced
from somewhere else and never prevent an object from being claimed by themselves. If that's
the only reference we're essentially not caching. In a micro test it might work as we might
not run into any GC.

At the very least we should use soft refs. Even there soft refs for caching are generally
not a good idea (there are some exceptions). They get cleaned up under heap pressure and when
the data is accessed again they need to be re-cached and add more heap pressure.

If we do not want to rock the boat now let's change them to soft refs, at least then we have
some caching. Otherwise let's use strong refs.



was (Author: lhofhansl):
+1

Weak refs are wrong anyway, they can only be used to access something that strongly referenced
from somewhere else and never prevent an object from being claimed. If that's the only reference
we're essentially not caching. In a micro test it might work as we might not run into any
GC.

At the very least we should use soft refs. Even there soft refs for caching are generally
not a good idea (there are some exceptions). They get cleaned up under heap pressure and when
the data is accessed again they need to be re-cached and add more heap pressure.

If we do not want to rock the boat now let's change them to soft refs, at least then we have
some caching. Otherwise let's use strong refs.


> Revisit table coprocessor classloader caching
> ---------------------------------------------
>
>                 Key: HBASE-12856
>                 URL: https://issues.apache.org/jira/browse/HBASE-12856
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 2.0.0, 0.98.10, 1.1.0
>
>
> For table coprocessors, we create coprocessor classloaders and cache them keyed on the
path to the coprocessor jar. However, we cache weak refs so it's quite possible if the refs
are not being retained due to heap pressure we might create a new classloader on the next
region open, i.e. after a split or relocation. If under very heavy write load, perhaps ingest
with all edits skipping the WAL for example, we'd be both generating a ton of garbage and
rapidly splitting at the same time. 
> We should revisit the notion of keeping only weak refs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message