hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8272) Erasure Coding: simplify the retry logic in DFSStripedInputStream
Date Wed, 29 Apr 2015 03:47:06 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14518671#comment-14518671

Zhe Zhang commented on HDFS-8272:

Thanks Jing for updating the patch! {{blockSeekTo}} looks good to me now. I also agree we
should get rid of retry in {{readWithStrategy}}.

The retry logic in {{DFSInputStream#readBuffer}} will try connecting to the same node:
        /* possibly retry the same node so that transient errors don't
         * result in application level failures (e.g. Datanode could have
         * closed the connection because the client is idle for too long).
         sourceFound = seekToBlockSource(pos);

I guess we should keep this part?

> Erasure Coding: simplify the retry logic in DFSStripedInputStream
> -----------------------------------------------------------------
>                 Key: HDFS-8272
>                 URL: https://issues.apache.org/jira/browse/HDFS-8272
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>         Attachments: h8272-HDFS-7285.000.patch, h8272-HDFS-7285.001.patch
> Currently in DFSStripedInputStream the retry logic is still the same with DFSInputStream.
More specifically, every failed read will try to search for another source node. And an exception
is thrown when no new source node can be identified. This logic is not appropriate for EC
inputstream and can be simplified.

This message was sent by Atlassian JIRA

View raw message