hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8272) Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read)
Date Wed, 29 Apr 2015 22:22:08 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14520406#comment-14520406
] 

Zhe Zhang commented on HDFS-8272:
---------------------------------

Thanks Jing for the update. I agree about removing {{seekToBlockSource}} at this stage. 

bq. With decoding functionality we do not need to spend more time trying our luck on the same
DN.
It's an interesting thought. Actually, it seems striped files should have a longer window
to retry the same DN than replicated files; since it's more expensive to decode than fetching
a replica from another DN.

bq. Currently calling seekToBlockSource will cause all the current block readers to be closed
(since blockSeekTo will call closeCurrentBlockReaders). To fix this we need to add extra complexity
so as to make sure only one block reader is retried.
bq. How about removing it by now and we can add it back if necessary in the future?
I agree. It's not straightforward to only retry one block reader. We should do it as a follow-on
task under HDFS-8031.

I will post my full review soon. 

> Erasure Coding: simplify the retry logic in DFSStripedInputStream (stateful read)
> ---------------------------------------------------------------------------------
>
>                 Key: HDFS-8272
>                 URL: https://issues.apache.org/jira/browse/HDFS-8272
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>         Attachments: HDFS-8272.002.patch, h8272-HDFS-7285.000.patch, h8272-HDFS-7285.001.patch
>
>
> Currently in DFSStripedInputStream the retry logic is still the same with DFSInputStream.
More specifically, every failed read will try to search for another source node. And an exception
is thrown when no new source node can be identified. This logic is not appropriate for EC
inputstream and can be simplified.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message