hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Error : java.lang.UnsatisfiedLinkError: failed to load the required native library for netty
Date Tue, 03 Oct 2017 15:33:00 GMT
nvm

I just retried up on linux and repro'd the below. Let me take a look....

Thanks,
S


On Tue, Oct 3, 2017 at 8:23 AM, Stack <stack@duboce.net> wrote:

> Thank you for the detail.
>
> Pardon the questions below asking for yet more detail. I am unable to
> reproduce locally or on another os (though we see this issue up on our
> build box).
>
> What is your OS when you see the below?
>
> On Tue, Oct 3, 2017 at 2:06 AM, Amit Kabra <amitkabraiiit@gmail.com>
> wrote:
>
>> Hello,
>>
>> I am using "branch-2" branch of hbase, when I run unit test I get
>> following
>> error for netty "java.lang.UnsatisfiedLinkError: failed to load the
>> required native library"
>>
>>
>> This is running a unit test in your eclipse environment?
>
> You are trying to run an hbase-spark unit test when you see the above?
>
>
>
>
>> *I already have following set in "maven-surefire-plugin" in pom.xml as
>> per http://hbase.apache.org/book.html#thirdparty
>> <http://hbase.apache.org/book.html#thirdparty>*
>>
>>
>>
>
> Are you embedding hbase into your application?
>
>
>
>>             <systemPropertyVariables>
>>                 <!--
>>               <test.build.classes>${test.bui
>> ld.classes}</test.build.classes>
>>                 -->
>>               <!--For shaded netty, to find the relocated .so.
>>                    Trick from
>>                 https://stackoverflow.com/questions/33825743/rename-
>> files-inside-a-jar-using-some-maven-plugin
>> <https://stackoverflow.com/questions/33825743/rename-files-inside-a-jar-using-some-maven-plugin>
>>
>>                 The netty jar has a .so in it. Shading requires rename of
>> the .so and then passing a system
>>                 property so netty finds the renamed .so and associates it
>> w/ the relocated netty files.
>>
>>                 The relocated netty is in hbase-thirdparty dependency.
>> Just
>> set this propery globally rather
>>                 than per module.
>>                -->
>>               <org.apache.hadoop.hbase.shaded.io.netty.packagePrefix>
>> org.apache.hadoop.hbase.shaded.</org.apache.hadoop.hbase.shaded.io.netty.
>> packagePrefix>
>>             </systemPropertyVariables>
>>
>>
>>
>> *And I see in the code as per HBASE-18271, all io.netty is already
>> replaced
>> with org.apache.hadoop.hbase.shaded.io.netty*
>>
>>
> The trailing period is also present?
>
>
>
>>
>> If I run a test from eclipse , I see the error immediately and my test
>> doesn't run, but when I run from command line , the test runs but I get
>> the
>> error at the end when the mvn command finishes.
>>
>>
>> Is it any eclipse test?
>
> Thank you. Let me try and fix this this morning.
>
> S
>
>
>
>
>
>> *Here is the complete error output.*
>>
>>
>>
>> [INFO]
>> [INFO] --- maven-surefire-plugin:2.19.1:test (default-test) @ hbase-spark
>> ---
>> [INFO]
>> [INFO] --- scalatest-maven-plugin:1.0:test (test) @ hbase-spark ---
>> Discovery starting.
>> Discovery completed in 1 second, 558 milliseconds.
>> Run starting. Expected test count is: 79
>> HBaseDStreamFunctionsSuite:
>> Formatting using clusterid: testClusterID
>> *** RUN ABORTED ***
>>   java.io.IOException: Shutting down
>>   at org.apache.hadoop.hbase.MiniHBaseCluster.init(
>> MiniHBaseCluster.java:232)
>>   at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(
>> MiniHBaseCluster.java:94)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(
>> HBaseTestingUtility.java:1124)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:1078)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:949)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:943)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:872)
>>   at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(
>> HBaseDStreamFunctionsSuite.scala:41)
>>   at org.scalatest.BeforeAndAfterAll$class.beforeAll(BeforeAndAfterAll.
>> scala:187)
>>   at org.apache.hadoop.hbase.spark.HBaseDStreamFunctionsSuite.beforeAll(
>> HBaseDStreamFunctionsSuite.scala:30)
>>   ...
>>   Cause: java.lang.RuntimeException: Failed construction of Master: class
>> org.apache.hadoop.hbase.master.HMasterorg.apache.
>> hadoop.hbase.shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM
>> ethods.epollin()I
>>   at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(
>> JVMClusterUtil.java:145)
>>   at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(
>> LocalHBaseCluster.java:217)
>>   at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(
>> LocalHBaseCluster.java:152)
>>   at org.apache.hadoop.hbase.MiniHBaseCluster.init(
>> MiniHBaseCluster.java:214)
>>   at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(
>> MiniHBaseCluster.java:94)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(
>> HBaseTestingUtility.java:1124)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:1078)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:949)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:943)
>>   at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(
>> HBaseTestingUtility.java:872)
>>   ...
>>   Cause: java.lang.UnsatisfiedLinkError: failed to load the required
>> native
>> library
>>   at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
>> Epoll.ensureAvailability(Epoll.java:78)
>>   at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
>> EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38)
>>   at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>(
>> NettyEventLoopGroupConfig.java:61)
>>   at org.apache.hadoop.hbase.regionserver.HRegionServer.<
>> init>(HRegionServer.java:552)
>>   at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475)
>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance(
>> NativeConstructorAccessorImpl.java:62)
>>   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>> DelegatingConstructorAccessorImpl.java:45)
>>   at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
>>   at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(
>> JVMClusterUtil.java:140)
>>   ...
>>   Cause: java.lang.UnsatisfiedLinkError: org.apache.hadoop.hbase.
>> shaded.io.netty.channel.epoll.NativeStaticallyReferencedJniM
>> ethods.epollin()I
>>   at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
>> NativeStaticallyReferencedJniMethods.epollin(Native Method)
>>   at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
>> Native.<clinit>(Native.java:66)
>>   at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
>> Epoll.<clinit>(Epoll.java:33)
>>   at org.apache.hadoop.hbase.shaded.io.netty.channel.epoll.
>> EpollEventLoopGroup.<clinit>(EpollEventLoopGroup.java:38)
>>   at org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.<init>(
>> NettyEventLoopGroupConfig.java:61)
>>   at org.apache.hadoop.hbase.regionserver.HRegionServer.<
>> init>(HRegionServer.java:552)
>>   at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:475)
>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> Method)
>>   at sun.reflect.NativeConstructorAccessorImpl.newInstance(
>> NativeConstructorAccessorImpl.java:62)
>>   at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
>> DelegatingConstructorAccessorImpl.java:45)
>>   ...
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache HBase ....................................... SUCCESS [
>> 1.575 s]
>> [INFO] Apache HBase - Checkstyle .......................... SUCCESS [
>> 0.317 s]
>> [INFO] Apache HBase - Annotations ......................... SUCCESS [
>> 0.537 s]
>> [INFO] Apache HBase - Build Configuration ................. SUCCESS [
>> 0.053 s]
>> [INFO] Apache HBase - Shaded Protocol ..................... SUCCESS [
>> 15.410 s]
>> [INFO] Apache HBase - Common .............................. SUCCESS [
>> 4.603 s]
>> [INFO] Apache HBase - Metrics API ......................... SUCCESS [
>> 1.213 s]
>> [INFO] Apache HBase - Hadoop Compatibility ................ SUCCESS [
>> 0.985 s]
>> [INFO] Apache HBase - Metrics Implementation .............. SUCCESS [
>> 0.863 s]
>> [INFO] Apache HBase - Hadoop Two Compatibility ............ SUCCESS [
>> 1.750 s]
>> [INFO] Apache HBase - Protocol ............................ SUCCESS [
>> 4.880 s]
>> [INFO] Apache HBase - Client .............................. SUCCESS [
>> 5.233 s]
>> [INFO] Apache HBase - Replication ......................... SUCCESS [
>> 1.040 s]
>> [INFO] Apache HBase - Prefix Tree ......................... SUCCESS [
>> 1.121 s]
>> [INFO] Apache HBase - Procedure ........................... SUCCESS [
>> 1.084 s]
>> [INFO] Apache HBase - Resource Bundle ..................... SUCCESS [
>> 0.092 s]
>> [INFO] Apache HBase - Server .............................. SUCCESS [
>> 19.849 s]
>> [INFO] Apache HBase - MapReduce ........................... SUCCESS [
>> 4.221 s]
>> [INFO] Apache HBase - Testing Util ........................ SUCCESS [
>> 3.273 s]
>> [INFO] Apache HBase - Thrift .............................. SUCCESS [
>> 5.519 s]
>> [INFO] Apache HBase - RSGroup ............................. SUCCESS [
>> 3.408 s]
>> [INFO] Apache HBase - Shell ............................... SUCCESS [
>> 3.859 s]
>> [INFO] Apache HBase - Coprocessor Endpoint ................ SUCCESS [
>> 4.038 s]
>> [INFO] Apache HBase - Backup .............................. SUCCESS [01:13
>> min]
>> [INFO] Apache HBase - Integration Tests ................... SUCCESS [
>> 4.229 s]
>> [INFO] Apache HBase - Examples ............................ SUCCESS [
>> 3.471 s]
>> [INFO] Apache HBase - Rest ................................ SUCCESS [
>> 4.448 s]
>> [INFO] Apache HBase - External Block Cache ................ SUCCESS [
>> 2.040 s]
>> [INFO] Apache HBase - Spark ............................... FAILURE [
>> 32.833 s]
>> [INFO] Apache HBase - Spark Integration Tests ............. SKIPPED
>> [INFO] Apache HBase - Assembly ............................ SKIPPED
>> [INFO] Apache HBase - Shaded .............................. SKIPPED
>> [INFO] Apache HBase - Shaded - Client ..................... SKIPPED
>> [INFO] Apache HBase - Shaded - MapReduce .................. SKIPPED
>> [INFO] Apache HBase Shaded Packaging Invariants ........... SKIPPED
>> [INFO] Apache HBase - Archetypes .......................... SKIPPED
>> [INFO] Apache HBase - Exemplar for hbase-client archetype . SKIPPED
>> [INFO] Apache HBase - Exemplar for hbase-shaded-client archetype SKIPPED
>> [INFO] Apache HBase - Archetype builder ................... SKIPPED
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] BUILD FAILURE
>> [INFO] ------------------------------------------------------------
>> ------------
>> [INFO] Total time: 03:26 min
>> [INFO] Finished at: 2017-09-27T19:34:35+05:30
>> [INFO] Final Memory: 345M/6055M
>> [INFO] ------------------------------------------------------------
>> ------------
>> [ERROR] Failed to execute goal org.scalatest:scalatest-maven-
>> plugin:1.0:test
>> (test) on project hbase-spark: There are test failures -> [Help 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with the
>> -e
>> switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/
>> MojoFailureException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR]   mvn <goals> -rf :hbase-spark
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message