hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7853) Erasure coding: extend LocatedBlocks to support reading from striped files
Date Wed, 04 Mar 2015 18:54:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347370#comment-14347370

Zhe Zhang commented on HDFS-7853:

Thanks Jing for the patch and sorry for the delayed review.

The overall structure looks good to me. The following implementation details are worth more
# You might have already started optimizing {{BlockInfoStriped#indices}} based on TODO msg.
Just a reminder that {{BlockInfoStripedUnderConstruction#blockIndices}} is only applicable
in over-replication as well. But if we get rid of indices for non-over-replicated {{replicas}},
we need to update {{addReplicaIfNotPresent}} as well, to always insert in the right position
# How about using a Map for those over-replicated replica (in addition to triplets and the
{{replicas}} array)? It's less efficient but should simplify the code. It's a rare condition
# If the non-over-replicated locations in {{LocatedBlockStriped}} are sorted, we can remove
the required {{indices}} filed in PB and add an optional field for the excess replicas and
their indices.

> Erasure coding: extend LocatedBlocks to support reading from striped files
> --------------------------------------------------------------------------
>                 Key: HDFS-7853
>                 URL: https://issues.apache.org/jira/browse/HDFS-7853
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Jing Zhao
>         Attachments: HDFS-7853.000.patch
> We should extend {{LocatedBlocks}} class so {{getBlockLocations}} can work with striping
layout (possibly an extra list specifying the index of each location in the group)

This message was sent by Atlassian JIRA

View raw message