Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Thu, 23 Apr 2015 00:15:39 +0000 (UTC)
From: "Zhe Zhang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12787192.1427843677000.665.1429748139733@Atlassian.JIRA>
In-Reply-To: <JIRA.12787192.1427843677000@Atlassian.JIRA>
References: <JIRA.12787192.1427843677000@Atlassian.JIRA>
 <JIRA.12787192.1427843677902@arcas>
Subject: [jira] [Commented] (HDFS-8033) Erasure coding: stateful
 (non-positional) read from files in striped layout
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14508196#comment-14508196 ] 

Zhe Zhang commented on HDFS-8033:
---------------------------------

Thanks Jing for the review; it's very helpful.

bq. Thus I guess we only need to define a new seekToNextBlock method which have the same functionality but different signature.
Good point. Besides {{readWithStrategy}}, {{blockSeekTo}} is only used in {{seekToBlockSource}} and {{seekToNewSource}} and these two are very simple methods. We should override them anyway.

bq. We can define a different version of readBuffer method which utilizes the reader list to read. And this new readBuffer method does not need to override the one in DFSInputStream.
bq. Another reason that we do not reuse the DFSInutStream#readBuffer is that the retry logic is different. From striped inputstream we do not need to try another source when hitting any failure.
I agree; a new {{readBuffer}} also makes it easier to implement parallel stateful read in the future.

[~xinwei] is working on HDFS-8201, which aims to add e2e tests with both striped input and output streams.

> Erasure coding: stateful (non-positional) read from files in striped layout
> ---------------------------------------------------------------------------
>
>                 Key: HDFS-8033
>                 URL: https://issues.apache.org/jira/browse/HDFS-8033
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8033.000.patch, HDFS-8033.001.patch, HDFS-8033.002.patch
>
>


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)