hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SammiChen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10935) Java-based EC codec does not reconstruct blocks correctly
Date Tue, 25 Oct 2016 07:22:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604480#comment-15604480

SammiChen commented on HDFS-10935:

Hi [~jojochuang], it's found out that this "Checksum mismatches" issue is because the second
checksum(the one with block reconstruct) is wrong.  The checksum is wrong is not because the
reconstructed block content is wrong. It's because the code reuses an existing {{DataChecksum}}
object while forgets to reset this checksum object state to initial state before using it.
 So the checksum is contaminated by its old state. I have upload the patch to address this

> Java-based EC codec does not reconstruct blocks correctly
> ---------------------------------------------------------
>                 Key: HDFS-10935
>                 URL: https://issues.apache.org/jira/browse/HDFS-10935
>             Project: Hadoop HDFS
>          Issue Type: Bug
>         Environment: JDK 1.8.0_91 on Mac OS X Yosemite 10.10.5
>            Reporter: Wei-Chiu Chuang
>            Assignee: SammiChen
>            Priority: Critical
> On my Mac, TestFileChecksum has been been failing since HDFS-10460. However, the jenkins
jobs have not reported the failures. Maybe it's an issue with my Mac or JDK.
> 9 out of 21 tests failed. 
> {noformat}
> java.lang.AssertionError: Checksum mismatches!
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery(TestFileChecksum.java:227)
> 	at org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery10(TestFileChecksum.java:336)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message