hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7613) Block placement policy for erasure coding groups
Date Tue, 14 Apr 2015 21:45:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14494969#comment-14494969

Zhe Zhang commented on HDFS-7613:

Thanks Walter for the patch! Please find my review below:

# It needs a rebase to resolve conflicts on {{BlockPlacementPolicyDefault}}
# A large portion of the current patch adds support for multiple placement policies ({{BlockPlacementPolicies}}).
We should separate this part out (maybe put it under HDFS-7068), and let this JIRA focus on
the EC policy itself. It will make the patches easier to review and also easier to backport
/ cherrypick.

# {{BlockPlacementPolicies}} in {{BlockManager}} looks good to me. If we are to support per-file
policy, we can't avoid maintaining a set of policies instead of a single {{blockplacement}}.
# Adding {{ecschema}} to the abstract method {{BlockPlacementPolicy#chooseTarget}} isn't ideal.
Maybe we should add a {{setSchema()}} method in the EC policy class instead?
# {{ErasureCodingWork}} does have access to the {{BlockCollection}}, so it can obtain the
{{isStriped}} flag from there, as well as the schema. So I don't think we need to extend its
signature to take the policy. I haven't checked all other places with extended signatures.
But in general, we can select the policy as late as possible, to avoid carrying it around
method signatures.

# The {{BlockPlacementPolicyEC}} name is probably fine, following the convention from {{BlockPlacementPolicyDefault}}.

The EC policy itself looks good overall. I will post a detailed review when above are addressed.

> Block placement policy for erasure coding groups
> ------------------------------------------------
>                 Key: HDFS-7613
>                 URL: https://issues.apache.org/jira/browse/HDFS-7613
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Walter Su
>         Attachments: HDFS-7613.001.patch
> Blocks in an erasure coding group should be placed in different failure domains -- different
DataNodes at the minimum, and different racks ideally.

This message was sent by Atlassian JIRA

View raw message