hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11964) RS-6-3-LEGACY has a decoding bug when it is used for pread
Date Sun, 27 Aug 2017 23:13:02 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143250#comment-16143250
] 

Kai Zheng commented on HDFS-11964:
----------------------------------

Hi [~tasanuma0829],

Thanks for working on this. I understand the issue and your fix. It looks to me {{int[] updateErasedIndexes(int[]
oldIndexes, int parityNum)}} is a post fix to the indexes array, I wonder if we could get
right at the first place. The indexes array is out by the following call, so the root cause
could be the inputs array isn't well set or initialized before doing the decode call, some
of the entries should be set null to indicate the positions are erased/corrupt or just not
to read.
{code}
    int[] erasedOrNotToReadIndexes =
        CoderUtil.getNullIndexes(decodingState.inputs);
{code}
Could you investigate more in your debugging session? We could also compare it with the default
RS coder. Thanks!

> RS-6-3-LEGACY has a decoding bug when it is used for pread
> ----------------------------------------------------------
>
>                 Key: HDFS-11964
>                 URL: https://issues.apache.org/jira/browse/HDFS-11964
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: erasure-coding
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Lei (Eddy) Xu
>            Assignee: Takanobu Asanuma
>         Attachments: HDFS-11964.1.patch
>
>
> TestDFSStripedInputStreamWithRandomECPolicy#testPreadWithDNFailure fails on trunk:
> {code}
> Running org.apache.hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
> Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 10.99 sec <<<
FAILURE! - in org.apache.hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
> testPreadWithDNFailure(org.apache.hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy)
 Time elapsed: 1.265 sec  <<< FAILURE!
> org.junit.internal.ArrayComparisonFailure: arrays first differed at element [327680];
expected:<-36> but was:<2>
> 	at org.junit.internal.ComparisonCriteria.arrayEquals(ComparisonCriteria.java:50)
> 	at org.junit.Assert.internalArrayEquals(Assert.java:473)
> 	at org.junit.Assert.assertArrayEquals(Assert.java:294)
> 	at org.junit.Assert.assertArrayEquals(Assert.java:305)
> 	at org.apache.hadoop.hdfs.TestDFSStripedInputStream.testPreadWithDNFailure(TestDFSStripedInputStream.java:306)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:498)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> {code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message