Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 10 Jul 2012 18:04:35 +0000 (UTC)
From: "Andrew Purtell (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <1273317577.29849.1341943475031.JavaMail.jiratomcat@issues-vm>
In-Reply-To: <1913664102.229.1341269818119.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Commented] (HBASE-6308) Coprocessors should be loaded in a
 custom ClassLoader to prevent dependency conflicts with HBase
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-6308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410638#comment-13410638 ] 

Andrew Purtell commented on HBASE-6308:
---------------------------------------

bq. Also, one of the reasons I didn't write a test in the original patch was that I couldn't seem to get the unit tests to run even without my changes. Most of the tests (including TestClassLoading) seem to fail with a NullPointerException in MiniDFSCluster.startDataNodes(), line 422. 

'umask 022' in the shell you have open for running tests, then run the suite again, James.
                
> Coprocessors should be loaded in a custom ClassLoader to prevent dependency conflicts with HBase
> ------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-6308
>                 URL: https://issues.apache.org/jira/browse/HBASE-6308
>             Project: HBase
>          Issue Type: Improvement
>          Components: coprocessors
>    Affects Versions: 0.92.1, 0.94.0
>            Reporter: James Baldassari
>             Fix For: 0.96.0
>
>         Attachments: 6308-v2.txt, HBASE-6308-0.92.patch, HBASE-6308-trunk.patch
>
>
> Currently each coprocessor is loaded with a URLClassLoader that puts the coprocessor's jar at the beginning of the classpath.  The URLClassLoader always tries to load classes from the parent ClassLoader first and only attempts to load from its own configured URLs if the class was not found by the parent.  This class loading behavior can be problematic for coprocessors that have common dependencies with HBase but whose versions are incompatible.  For example, I have a coprocessor that depends on a different version of Avro than the version used by HBase.  The current class loading behavior results in NoSuchMethodErrors in my coprocessor because some Avro classes have already been loaded by HBase, and the ClassLoader for my coprocessor picks up HBase's loaded classes first.
> My proposed solution to this problem is to use a custom ClassLoader when instantiating coprocessor instances.  This custom ClassLoader would always attempt to load classes from the coprocessor's jar first and would only delegate to the parent ClassLoader if the class were not found in the coprocessor jar.  However, certain classes would need to be exempt from this behavior.  As an example, if the Copcoessor interface were loaded by both the region server's ClassLoader and the coprocessor's custom ClassLoader, then the region server would get a ClassCastException when attempting to cast the coprocessor instance to the Coprocessor interface.  This problem can be avoided by defining a set of class name prefixes that would be exempt from loading by the custom ClassLoader.  When loading a class, if the class starts with any of these prefixes (e.g. "org.apache.hadoop"), then the ClassLoader would delegate immediately to the parent ClassLoader.
> I've already implemented a patch to provide this functionality which I'll attach shortly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira