hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6308) Coprocessors should be loaded in a custom ClassLoader to prevent dependency conflicts with HBase
Date Tue, 10 Jul 2012 18:04:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410638#comment-13410638
] 

Andrew Purtell commented on HBASE-6308:
---------------------------------------

bq. Also, one of the reasons I didn't write a test in the original patch was that I couldn't
seem to get the unit tests to run even without my changes. Most of the tests (including TestClassLoading)
seem to fail with a NullPointerException in MiniDFSCluster.startDataNodes(), line 422. 

'umask 022' in the shell you have open for running tests, then run the suite again, James.
                
> Coprocessors should be loaded in a custom ClassLoader to prevent dependency conflicts
with HBase
> ------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6308
>                 URL: https://issues.apache.org/jira/browse/HBASE-6308
>             Project: HBase
>          Issue Type: Improvement
>          Components: coprocessors
>    Affects Versions: 0.92.1, 0.94.0
>            Reporter: James Baldassari
>             Fix For: 0.96.0
>
>         Attachments: 6308-v2.txt, HBASE-6308-0.92.patch, HBASE-6308-trunk.patch
>
>
> Currently each coprocessor is loaded with a URLClassLoader that puts the coprocessor's
jar at the beginning of the classpath.  The URLClassLoader always tries to load classes from
the parent ClassLoader first and only attempts to load from its own configured URLs if the
class was not found by the parent.  This class loading behavior can be problematic for coprocessors
that have common dependencies with HBase but whose versions are incompatible.  For example,
I have a coprocessor that depends on a different version of Avro than the version used by
HBase.  The current class loading behavior results in NoSuchMethodErrors in my coprocessor
because some Avro classes have already been loaded by HBase, and the ClassLoader for my coprocessor
picks up HBase's loaded classes first.
> My proposed solution to this problem is to use a custom ClassLoader when instantiating
coprocessor instances.  This custom ClassLoader would always attempt to load classes from
the coprocessor's jar first and would only delegate to the parent ClassLoader if the class
were not found in the coprocessor jar.  However, certain classes would need to be exempt from
this behavior.  As an example, if the Copcoessor interface were loaded by both the region
server's ClassLoader and the coprocessor's custom ClassLoader, then the region server would
get a ClassCastException when attempting to cast the coprocessor instance to the Coprocessor
interface.  This problem can be avoided by defining a set of class name prefixes that would
be exempt from loading by the custom ClassLoader.  When loading a class, if the class starts
with any of these prefixes (e.g. "org.apache.hadoop"), then the ClassLoader would delegate
immediately to the parent ClassLoader.
> I've already implemented a patch to provide this functionality which I'll attach shortly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message