hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sree Vaddi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-15716) native library dependency on the very first build
Date Mon, 03 Sep 2018 15:56:00 GMT
Sree Vaddi created HADOOP-15716:
-----------------------------------

             Summary: native library dependency on the very first build
                 Key: HADOOP-15716
                 URL: https://issues.apache.org/jira/browse/HADOOP-15716
             Project: Hadoop Common
          Issue Type: Bug
          Components: common
    Affects Versions: 3.2.0
         Environment: [INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: osx
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.13
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 13
[INFO] os.detected.classifier: osx-x86_64
            Reporter: Sree Vaddi


When building hadoop (hdds exactly, but hadoop, too) for the very first time, Tests fails
due to the dependency on the native lib (missing libhadoop.so).  As a work around, one can
get past by skipping tests.  But it sounds chicken & egg situation, to have installed
'libhadoop.so' before building hadoop for the very first time.

 

Suggestion to have a first time flag or some logic figure it, then skip the failing tests
and/or compile/install libhadoop.so before running those failing tests.

 

 

HW14169:hadoop svaddi$ mvn clean package install -Phdds -Pdist -Dtar


[INFO] Running org.apache.hadoop.util.TestTime
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.06 s - in org.apache.hadoop.util.TestTime
[INFO] Running org.apache.hadoop.util.TestNativeCodeLoader
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.117 s <<<
FAILURE! - in org.apache.hadoop.util.TestNativeCodeLoader
[ERROR] testNativeCodeLoaded(org.apache.hadoop.util.TestNativeCodeLoader)  Time elapsed:
0.027 s  <<< FAILURE!
java.lang.AssertionError: TestNativeCodeLoader: libhadoop.so testing was required, but libhadoop.so
was not loaded.
    at org.junit.Assert.fail(Assert.java:88)
    at org.apache.hadoop.util.TestNativeCodeLoader.testNativeCodeLoaded(TestNativeCodeLoader.java:48)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)

[INFO] Running org.apache.hadoop.util.TestLightWeightCache
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.516 s - in org.apache.hadoop.util.TestLightWeightCache

[INFO] Running org.apache.hadoop.io.compress.lz4.TestLz4CompressorDecompressor
[WARNING] Tests run: 13, Failures: 0, Errors: 0, Skipped: 13, Time elapsed: 0.128 s - in org.apache.hadoop.io.compress.lz4.TestLz4CompressorDecompressor
[INFO] Running org.apache.hadoop.io.compress.TestCodec
[ERROR] Tests run: 26, Failures: 1, Errors: 0, Skipped: 5, Time elapsed: 55.533 s <<<
FAILURE! - in org.apache.hadoop.io.compress.TestCodec
[ERROR] testCodecPoolCompressorReinit(org.apache.hadoop.io.compress.TestCodec)  Time elapsed:
0.031 s  <<< FAILURE!
java.lang.AssertionError: Compressed bytes contrary to configuration
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.assertTrue(Assert.java:41)
    at org.apache.hadoop.io.compress.TestCodec.gzipReinitTest(TestCodec.java:431)
    at org.apache.hadoop.io.compress.TestCodec.testCodecPoolCompressorReinit(TestCodec.java:502)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
    at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)

[INFO] Running org.apache.hadoop.io.compress.zlib.TestZlibCompressorDecompressor
[WARNING] Tests run: 10, Failures: 0, Errors: 0, Skipped: 10, Time elapsed: 0.293 s - in org.apache.hadoop.io.compress.zlib.TestZlibCompressorDecompressor


[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.371 s - in org.apache.hadoop.ipc.TestProtoBufRpc
[INFO] Running org.apache.hadoop.ipc.TestIPC
[ERROR] Tests run: 39, Failures: 0, Errors: 2, Skipped: 1, Time elapsed: 88.727 s <<<
FAILURE! - in org.apache.hadoop.ipc.TestIPC
[ERROR] testHttpGetResponse(org.apache.hadoop.ipc.TestIPC)  Time elapsed: 0.016 s  <<<
ERROR!
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.net.SocketInputStream.read(SocketInputStream.java:127)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
    at org.apache.hadoop.ipc.TestIPC.doIpcVersionTest(TestIPC.java:1579)
    at org.apache.hadoop.ipc.TestIPC.testHttpGetResponse(TestIPC.java:1074)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

[ERROR] testIpcFromHadoop_0_18_13(org.apache.hadoop.ipc.TestIPC)  Time elapsed: 0.009 s 
<<< ERROR!
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:210)
    at java.net.SocketInputStream.read(SocketInputStream.java:141)
    at java.net.SocketInputStream.read(SocketInputStream.java:127)
    at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:100)
    at org.apache.hadoop.ipc.TestIPC.doIpcVersionTest(TestIPC.java:1579)
    at org.apache.hadoop.ipc.TestIPC.testIpcFromHadoop_0_18_13(TestIPC.java:1056)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

[INFO] Running org.apache.hadoop.ipc.TestRPCWaitForProxy
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 11.31 s - in org.apache.hadoop.ipc.TestRPCWaitForProxy


[INFO] Running org.apache.hadoop.fs.TestRawLocalFileSystemContract
[ERROR] Tests run: 44, Failures: 0, Errors: 1, Skipped: 18, Time elapsed: 0.981 s <<<
FAILURE! - in org.apache.hadoop.fs.TestRawLocalFileSystemContract
[ERROR] testPermission(org.apache.hadoop.fs.TestRawLocalFileSystemContract)  Time elapsed:
0.296 s  <<< ERROR!
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Ljava/lang/String;)Lorg/apache/hadoop/io/nativeio/NativeIO$POSIX$Stat;
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.stat(Native Method)
    at org.apache.hadoop.io.nativeio.NativeIO$POSIX.getStat(NativeIO.java:451)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfoByNativeIO(RawLocalFileSystem.java:821)
    at org.apache.hadoop.fs.TestRawLocalFileSystemContract.testPermission(TestRawLocalFileSystemContract.java:112)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)

[INFO] Running org.apache.hadoop.fs.TestFsShellTouch
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.492 s - in org.apache.hadoop.fs.TestFsShellTouch


[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.35 s - in org.apache.hadoop.conf.TestCommonConfigurationFields
[INFO] Running org.apache.hadoop.conf.TestConfigRedactor
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.182 s - in org.apache.hadoop.conf.TestConfigRedactor
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   TestCodec.testCodecPoolCompressorReinit:502->gzipReinitTest:431 Compressed
bytes contrary to configuration
[ERROR]   TestNativeCodeLoader.testNativeCodeLoaded:48 TestNativeCodeLoader: libhadoop.so
testing was required, but libhadoop.so was not loaded.
[ERROR] Errors:
[ERROR]   TestRawLocalFileSystemContract.testPermission:112 » UnsatisfiedLink org.apache...
[ERROR]   TestIPC.testHttpGetResponse:1074->doIpcVersionTest:1579 » Socket Connection
re...
[ERROR]   TestIPC.testIpcFromHadoop_0_18_13:1056->doIpcVersionTest:1579 » Socket Connect...
[INFO]
[ERROR] Tests run: 4130, Failures: 2, Errors: 3, Skipped: 358
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop Main 3.2.0-SNAPSHOT .................. SUCCESS [  1.428 s]
[INFO] Apache Hadoop Build Tools .......................... SUCCESS [  2.570 s]
[INFO] Apache Hadoop Project POM .......................... SUCCESS [  1.791 s]
[INFO] Apache Hadoop Annotations .......................... SUCCESS [  4.855 s]
[INFO] Apache Hadoop Assemblies ........................... SUCCESS [  1.197 s]
[INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [  2.706 s]
[INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [  7.225 s]
[INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 12.220 s]
[INFO] Apache Hadoop Auth ................................. SUCCESS [02:22 min]
[INFO] Apache Hadoop Auth Examples ........................ SUCCESS [  4.535 s]
[INFO] Apache Hadoop Common ............................... FAILURE [26:23 min]
[INFO] Apache Hadoop NFS .................................. SKIPPED
[INFO] Apache Hadoop KMS .................................. SKIPPED
...
...
...
[INFO] Apache Hadoop Cloud Storage Project ................ SKIPPED
[INFO] Apache Hadoop Ozone Acceptance Tests 3.2.0-SNAPSHOT  SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29:27 min
[INFO] Finished at: 2018-09-02T06:00:21-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test
(default-test) on project hadoop-common: There are test failures.
[ERROR]
[ERROR] Please refer to /Users/svaddi/SreeVaddi/sources/github/sreev/hadoop/hadoop-common-project/hadoop-common/target/surefire-reports
for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream
and [date]-jvmRun[N].dumpstream.
[ERROR] -> [Help 1]
[ERROR]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org


Mime
View raw message