hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6589) RegionServer can't load class for dynamically loaded coprocessors with self defined class
Date Mon, 24 Nov 2014 07:46:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222755#comment-14222755

Hao Chen commented on HBASE-6589:

I found that the problem is not about the coprocessor framework but the RPC system, because
If adding below static code in the coprocessor implementation, we found that it should load
successfully, which means: these customized classes were loaded in Coprocoessor but not loaded
by HbaseObjectWritable. As to the reason is that coprocessor classes are loaded by custom
CoprocessorClassLoader instead of the region server's default ClassLoader, so custom classes
using in coprocessor are only loaded for coprocessor but not for RPC system, and RPC system
is just like gateway before calling Coprocessor instance. 

One possible solution is that you may serialize the customized class into any type that HbaseObjectWritable
recognize like byte[] in client, pass to coprocessor, and then deserializewith customized
class  in coprocessor side.

       try {
	      LOG.info("Loading class CustomClass");
              HReportsServerProtocol .class.getClassLoader().loadClass("CustomClass ");
              LOG.info("Loaded class CustomClass");
        } catch (ClassNotFoundException e) {
	       LOG.error("Failed to load class: CustomClass", e);

// == log output == 
// INFO Loading class CustomClass
// INFO Loaded class CustomClass

> RegionServer can't load class for dynamically loaded coprocessors with self defined class
> -----------------------------------------------------------------------------------------
>                 Key: HBASE-6589
>                 URL: https://issues.apache.org/jira/browse/HBASE-6589
>             Project: HBase
>          Issue Type: Bug
>          Components: Coprocessors, regionserver
>            Reporter: ShiXing
> When using coprocessor with custom classes like LongColumnInterpreter(mine is MultiColumnSchema),
the coprocessor can not work for hot deploy, if the custom classes do not deploy in the regionserver's
classpath. Although the self-defined class is deployed in the regions' classpath through hdfs
> The exception threw at the regionserver's log:
> {code}
> 2012-08-15 16:24:24,403 ERROR org.apache.hadoop.hbase.io.HbaseObjectWritable: Error in
> java.io.IOException: Can't find class com.taobao.hbase.coprocessor.MultiColumnSchema
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:674)
>         at org.apache.hadoop.hbase.client.coprocessor.Exec.readFields(Exec.java:114)
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:682)
>         at org.apache.hadoop.hbase.ipc.Invocation.readFields(Invocation.java:125)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:1292)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:1207)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:735)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:524)
>         at org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:499)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.ClassNotFoundException: com.taobao.hbase.coprocessor.MultiColumnSchema
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>         at java.lang.Class.forName0(Native Method)
>         at java.lang.Class.forName(Class.java:247)
>         at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:943)
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.getClassByName(HbaseObjectWritable.java:784)
>         at org.apache.hadoop.hbase.io.HbaseObjectWritable.readObject(HbaseObjectWritable.java:671)
>         ... 11 more
> {code} 
> It is similar as HBASE-4946, but I do not know how to solve this bug.
> If add these custom class to the RegionServer's classloader may fix it, but it is conflicted
with HBASE-6308 to prevent dependency conflicts.
> Does anyone have some idea?

This message was sent by Atlassian JIRA

View raw message