hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SammiChen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14479) Erasurecode testcase failures with native enabled
Date Wed, 28 Jun 2017 22:00:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16067320#comment-16067320
] 

SammiChen commented on HADOOP-14479:
------------------------------------

First of all, these three failed unit tests are not related with ISA-L version change. There
are actually hidden issues.
1.TestHHXORErasureCoder
The implementation of native XOR encoder/decoder has array out of index issue which cause
the JVM crash.
2.TestRSErasureCoder
One failed unit test is to test underlying encoder reuse by execution testCoding twice, while
the underlying encoder is released explicitly by ErasureEncodingStep.finish. By review the
context, I think ErasureEncodingStep.finish is not the right place to release the encoder.
3. TestCodecRawCoderMapping
One failed unit test is because it doesn't consider the native RS coder situation.

I have tried the patch locally.
[~Ayappan], thanks for reporting this. You can have a try with the new patch. 

Besides, I will close HADOOP-14593 later since I have merged fixes for 3 unit cases into one
patch. 

> Erasurecode testcase failures with native enabled
> -------------------------------------------------
>
>                 Key: HADOOP-14479
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14479
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common
>    Affects Versions: 3.0.0-alpha3
>         Environment: x86_64 Ubuntu 16.04.02 LTS
>            Reporter: Ayappan
>            Assignee: SammiChen
>            Priority: Critical
>              Labels: hdfs-ec-3.0-must-do
>         Attachments: HADOOP-14479.001.patch
>
>
> I built hadoop with ISA-L support. I took the ISA-L code from https://github.com/01org/isa-l
 (tag v2.18.0) and built it. While running the UTs , following three testcases are failing
> 1)TestHHXORErasureCoder
> Tests run: 7, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 1.106 sec <<<
FAILURE! - in org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder
> testCodingDirectBuffer_10x4_erasing_p1(org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder)
 Time elapsed: 0.029 sec  <<< FAILURE!
> java.lang.AssertionError: Decoding and comparing failed.
>         at org.junit.Assert.fail(Assert.java:88)
>         at org.junit.Assert.assertTrue(Assert.java:41)
>         at org.apache.hadoop.io.erasurecode.TestCoderBase.compareAndVerify(TestCoderBase.java:170)
>         at org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.compareAndVerify(TestErasureCoderBase.java:141)
>         at org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.performTestCoding(TestErasureCoderBase.java:98)
>         at org.apache.hadoop.io.erasurecode.coder.TestErasureCoderBase.testCoding(TestErasureCoderBase.java:69)
>         at org.apache.hadoop.io.erasurecode.coder.TestHHXORErasureCoder.testCodingDirectBuffer_10x4_erasing_p1(TestHHXORErasureCoder.java:64)
> 2)TestRSErasureCoder
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.591 sec - in org.apache.hadoop.io.erasurecode.coder.TestXORCoder
> Running org.apache.hadoop.io.erasurecode.coder.TestRSErasureCoder
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x00007f486a28a6e4, pid=8970, tid=0x00007f4850927700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_121-b13) (build 1.8.0_121-8u121-b13-0ubuntu1.16.04.2-b13)
> # Java VM: OpenJDK 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops)
> # Problematic frame:
> # C  [libc.so.6+0x8e6e4]
> #
> # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try
"ulimit -c unlimited" before starting Java again
> #
> # An error report file with more information is saved as:
> # /home/ayappan/hadoop/hadoop-common-project/hadoop-common/hs_err_pid8970.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> # The crash happened outside the Java Virtual Machine in native code.
> # See problematic frame for where to report the bug.
> #
> 3)TestCodecRawCoderMapping
> Running org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.559 sec <<<
FAILURE! - in org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping
> testRSDefaultRawCoder(org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping)  Time
elapsed: 0.015 sec  <<< FAILURE!
> java.lang.AssertionError: null
>         at org.junit.Assert.fail(Assert.java:86)
>         at org.junit.Assert.assertTrue(Assert.java:41)
>         at org.junit.Assert.assertTrue(Assert.java:52)
>         at org.apache.hadoop.io.erasurecode.TestCodecRawCoderMapping.testRSDefaultRawCoder(TestCodecRawCoderMapping.java:58)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message