hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12856) Revisit table coprocessor classloader caching
Date Wed, 14 Jan 2015 21:24:34 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14277713#comment-14277713

Andrew Purtell commented on HBASE-12856:

Lars and I discussed this issue some more. Having a weak reference to the classloader in the
map is fine, the values will point to CoprocessorClassLoader instances that should never be
GCed because we don't unload coprocessor classes. The CoprocessorClassLoader instances will
be strongly referenced by every java.lang.Class they instantiated, indefinitely.

Even then, we really don't want the "classLoadersCache" map in CoprocessorClassLoader to be
a cache, we want it to be an index. If/when we implement coprocessor class unloading we wouldn't
want a strong reference in this map retaining the classloader. The comments around this field
and even its name could be reconsidered, 'cache' -> 'index'.

It seems fine to leave this as is. However I will leave this issue open for a bit in case
we are mistaken in some way or there is additional comment or suggestion.

> Revisit table coprocessor classloader caching
> ---------------------------------------------
>                 Key: HBASE-12856
>                 URL: https://issues.apache.org/jira/browse/HBASE-12856
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 2.0.0, 0.98.10, 1.1.0
> For table coprocessors, we create coprocessor classloaders and cache them keyed on the
path to the coprocessor jar. However, we cache weak refs so it's quite possible if the refs
are not being retained due to heap pressure we might create a new classloader on the next
region open, i.e. after a split or relocation. If under very heavy write load, perhaps ingest
with all edits skipping the WAL for example, we'd be both generating a ton of garbage and
rapidly splitting at the same time. 
> We should revisit the notion of keeping only weak refs.

This message was sent by Atlassian JIRA

View raw message