hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8033) Erasure coding: stateful (non-positional) read from files in striped layout
Date Tue, 21 Apr 2015 20:53:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14505735#comment-14505735

Jing Zhao commented on HDFS-8033:

Thanks for working on this, Zhe! And thanks for the review, Yi!

Besides Yi's comments, some other thoughts:
# In {{DFSInputStream}}, the reading state can be captured by {{blockReader}}, {{currentNode}},
etc. When reading striped blocks, it may be more clear to define the state as the group of
readers and DataNodes.
# Besides, the current patch mainly overrides the {{blockSeekTo}} and the {{readBuffer}} methods.
In this way, it may be hard to support parallel reading or failure handling (both will be
handled in separate jiras) in the future.
# Thus instead of declare {{blockReader}}, {{currentNode}}, etc as protected and keep updating
them in {{DFSStripedInputStream}}, I guess it may be better to define {{DFSStripedInputStream}}'s
own states (using a list of readers etc.). We may want to override the whole {{readWithStrategy}}
method in {{DFSStripedInputStream}} as well.

> Erasure coding: stateful (non-positional) read from files in striped layout
> ---------------------------------------------------------------------------
>                 Key: HDFS-8033
>                 URL: https://issues.apache.org/jira/browse/HDFS-8033
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8033.000.patch, HDFS-8033.001.patch

This message was sent by Atlassian JIRA

View raw message