hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SammiChen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10935) Java-based EC codec does not reconstruct blocks correctly
Date Tue, 25 Oct 2016 07:22:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15604480#comment-15604480
] 

SammiChen commented on HDFS-10935:
----------------------------------

Hi [~jojochuang], it's found out that this "Checksum mismatches" issue is because the second
checksum(the one with block reconstruct) is wrong.  The checksum is wrong is not because the
reconstructed block content is wrong. It's because the code reuses an existing {{DataChecksum}}
object while forgets to reset this checksum object state to initial state before using it.
 So the checksum is contaminated by its old state. I have upload the patch to address this
issue. 

> Java-based EC codec does not reconstruct blocks correctly
> ---------------------------------------------------------
>
>                 Key: HDFS-10935
>                 URL: https://issues.apache.org/jira/browse/HDFS-10935
>             Project: Hadoop HDFS
>          Issue Type: Bug
>         Environment: JDK 1.8.0_91 on Mac OS X Yosemite 10.10.5
>            Reporter: Wei-Chiu Chuang
>            Assignee: SammiChen
>            Priority: Critical
>
> On my Mac, TestFileChecksum has been been failing since HDFS-10460. However, the jenkins
jobs have not reported the failures. Maybe it's an issue with my Mac or JDK.
> 9 out of 21 tests failed. 
> {noformat}
> java.lang.AssertionError: Checksum mismatches!
> 	at org.junit.Assert.fail(Assert.java:88)
> 	at org.junit.Assert.assertTrue(Assert.java:41)
> 	at org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery(TestFileChecksum.java:227)
> 	at org.apache.hadoop.hdfs.TestFileChecksum.testStripedFileChecksumWithMissedDataBlocksRangeQuery10(TestFileChecksum.java:336)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message