hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7853) Erasure coding: extend LocatedBlocks to support reading from striped files
Date Thu, 05 Mar 2015 01:02:19 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347868#comment-14347868

Jing Zhao commented on HDFS-7853:

Thanks for the review, Zhe!

bq. Just a reminder that BlockInfoStripedUnderConstruction#blockIndices is only applicable
in over-replication as well.

This may not be true. For a newly created BlockInfoStripedUC, {{blockIndices}} is not useful
since we simply assign block (in the group) to storage according to the target storage sequence.
However, if the namenode restarts before the block is completed, the replicas array is later
rebuilt based on the incoming block reports, and we need to record the corresponding block
index somewhere (or keep some null elements in the ReplicaUC array).

bq. If the non-over-replicated locations in LocatedBlockStriped are sorted

Here a sorted array may not be enough, e.g., if we miss several blocks in the middle. Then
we may still need an extra index array in the msg.

bq. blk.setBlockId(bg.getBlock().getBlockId() + i);

I guess here the code should be "... + j);" ?

> Erasure coding: extend LocatedBlocks to support reading from striped files
> --------------------------------------------------------------------------
>                 Key: HDFS-7853
>                 URL: https://issues.apache.org/jira/browse/HDFS-7853
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Jing Zhao
>         Attachments: HDFS-7853.000.patch
> We should extend {{LocatedBlocks}} class so {{getBlockLocations}} can work with striping
layout (possibly an extra list specifying the index of each location in the group)

This message was sent by Atlassian JIRA

View raw message