hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16264) Figure how to deal with endpoints and shaded pb
Date Wed, 31 Aug 2016 06:05:21 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451276#comment-15451276
] 

stack commented on HBASE-16264:
-------------------------------

Digging in more, the above NoClassDefFoundError is a linkage time error (parent class is LinkageError).
Reading the JVM Spec.[2], we could be getting this exception at linking time or probably later,
at resolve when we are trying replace the Class constant_pool 'logical' references with actual
locations (the discussion about late vs eager resolve doesn't seem to apply here; our violation
must be triggering eager error reporting). In the Class file, it has methods which have com.google.protobuf.*
parameters (for Coprocessor Endpoints). We're messing stuff up when Classes are uniquely identified
by the name+classloader.

In the above, we are just trying to get the public methods on the class looking for the main
method, one of the first tasks getting a program off the ground. I tried moving the HRegionServer
main to its own class and starting the program from there but just got a different variation
on the above; it too has to load HRegionServer class eventually with similar repercussions:

1408 RCL loading relocated com.google.protobuf.RpcCallback as "org.apache.hadoop.hbase.shaded.com.google.protobuf.RpcCallback"
1409 [Loaded org.apache.hadoop.hbase.shaded.com.google.protobuf.RpcCallback from file:/Users/stack/checkouts/hbase.git/hbase-shaded-libs/target/hbase-shaded-libs-2.0.0-SNAPSHOT.jar]
1410 Exception in thread "main" [Loaded java.lang.Throwable$PrintStreamOrWriter from /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/rt.jar]
1411 [Loaded java.lang.Throwable$WrappedPrintStream from /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/rt.jar]
1412 [Loaded java.util.IdentityHashMap$KeySet from /Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home/jre/lib/rt.jar]
1413 java.lang.NoClassDefFoundError: com/google/protobuf/RpcCallback
1414   at java.lang.Class.forName0(Native Method)
1415   at java.lang.Class.forName(Class.java:264)
1416   at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.main(HRegionServerCommandLine.java:107)

getDeclaredMethods0 and forName0 are native methods [3] NoClassDefFoundError is thrown if
we can't 'fix' the classname -- see end of [4] -- or it is coming up out of JVM_FindClassFromClassLoader
[5]. I kinda stopped trying to chase at this point.

Next up I'd say would be rewriting byte codes to replace any com.google.protobuf references
with org.apache.hadoop.hbase.shaded.com.google.protobuf using something like ASM -- it has
a means for doing this on load called RemappingClassAdapter -- but I'd have to do it for every
class loaded. This seems over the top.

I'm going to go back to a modified tactic #1. I'll fill out the backing doc w/ new approach
which I'll call tactic #3 where I move the embedded endpoints after hbase-client and hbase-server....

1. http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8u40-b25/sun/launcher/LauncherHelper.java#LauncherHelper.validateMainClass%28java.lang.Class%29
2.https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-5.html#jvms-5.4
3. http://code.metager.de/source/xref/openjdk/jdk8/jdk/src/share/native/java/lang/Class.c#99
4. https://github.com/frohoff/jdk7u/blob/master/src/share/native/common/check_format.c
5. http://hg.openjdk.java.net/jdk7/jdk7/hotspot/file/tip/src/share/vm/prims/jvm.cpp#l737

> Figure how to deal with endpoints and shaded pb
> -----------------------------------------------
>
>                 Key: HBASE-16264
>                 URL: https://issues.apache.org/jira/browse/HBASE-16264
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Coprocessors, Protobufs
>            Reporter: stack
>            Assignee: stack
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: 16264.tactic2.patch, HBASE-16264.master.001.patch, HBASE-16264.master.002.patch,
HBASE-16264.master.003.patch, HBASE-16264.master.004.patch, HBASE-16264.master.005.patch,
HBASE-16264.master.006.patch, HBASE-16264.master.007.patch
>
>
> Come up w/ a migration plan for coprocessor endpoints when our pb is shaded. Would be
sweet if could make it so all just worked. At worst, come up w/ a prescription for how to
migrate existing CPs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message