hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature
Date Tue, 18 Aug 2015 20:20:46 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14701924#comment-14701924
] 

Zhe Zhang commented on HDFS-8909:
---------------------------------

Thanks Jing for the work! Most of the patch is clean refactoring and LGTM. A few issues:
# Do we still need a separate {{ReplicaUnderConstruction}}? If so should we refactor in trunk?
# In {{BlockUnderConstructionFeature#addReplicaIfNotPresent}}, a contiguous block should not
have different IDs reported. Should we add some assertion to be more clear? I think {{addReplicaIfNotPresent}}
and {{setExpectedLocations}} are the only methods requiring different contiguous/striped logics.

{code}
        if (expected == storage) {
          replicas[i].setBlockId(reportedBlock.getBlockId());
          replicas[i].setGenerationStamp(reportedBlock.getGenerationStamp());
{code}
# We are exposing the striping-only interface {{getBlockIndices}} in {{BlockUnderConstructionFeature}}.
This is a similar concern as Haohui [commented | https://issues.apache.org/jira/browse/HDFS-8801?focusedCommentId=14699936&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14699936]
under HDFS-8801. We can address as follow-on.
# A few comments and debugging messages in {{BlockUnderConstructionFeature}} still mention
"striped"
# Please see if you want to simplify {{BlockInfo#convertToBlockUnderConstruction}} as below:
{code}
  public void convertToBlockUnderConstruction(BlockUCState s,
      DatanodeStorageInfo[] targets) {
    if (isComplete()) {
      uc = new BlockUnderConstructionFeature(this, s, targets);
    } else {
      // the block is already under construction
      uc.setBlockUCState(s);
    }
    uc.setExpectedLocations(this, targets, this.isStriped());
  }
{code}
By doing that we can also simplify {{BlockUnderConstructionFeature}} constructor, removing
{{isStriped}} and {{targets}} from signature. 

> Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature
> --------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8909
>                 URL: https://issues.apache.org/jira/browse/HDFS-8909
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: HDFS-7285
>            Reporter: Zhe Zhang
>            Assignee: Jing Zhao
>         Attachments: HDFS-8909.000.patch
>
>
> HDFS-8801 converts {{BlockInfoUC}} as a feature. We should consolidate {{BlockInfoContiguousUC}}
and {{BlockInfoStripedUC}} logics to use this feature.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message