hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8033) Erasure coding: stateful (non-positional) read from files in striped layout
Date Thu, 23 Apr 2015 00:15:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14508196#comment-14508196

Zhe Zhang commented on HDFS-8033:

Thanks Jing for the review; it's very helpful.

bq. Thus I guess we only need to define a new seekToNextBlock method which have the same functionality
but different signature.
Good point. Besides {{readWithStrategy}}, {{blockSeekTo}} is only used in {{seekToBlockSource}}
and {{seekToNewSource}} and these two are very simple methods. We should override them anyway.

bq. We can define a different version of readBuffer method which utilizes the reader list
to read. And this new readBuffer method does not need to override the one in DFSInputStream.
bq. Another reason that we do not reuse the DFSInutStream#readBuffer is that the retry logic
is different. From striped inputstream we do not need to try another source when hitting any
I agree; a new {{readBuffer}} also makes it easier to implement parallel stateful read in
the future.

[~xinwei] is working on HDFS-8201, which aims to add e2e tests with both striped input and
output streams.

> Erasure coding: stateful (non-positional) read from files in striped layout
> ---------------------------------------------------------------------------
>                 Key: HDFS-8033
>                 URL: https://issues.apache.org/jira/browse/HDFS-8033
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8033.000.patch, HDFS-8033.001.patch, HDFS-8033.002.patch

This message was sent by Atlassian JIRA

View raw message