Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Thu, 23 Jul 2015 21:57:04 +0000 (UTC)
From: "Zhe Zhang (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12833908.1432939634000.275475.1437688624788@Atlassian.JIRA>
In-Reply-To: <JIRA.12833908.1432939634000@Atlassian.JIRA>
References: <JIRA.12833908.1432939634000@Atlassian.JIRA>
 <JIRA.12833908.1432939634021@arcas>
Subject: [jira] [Commented] (HDFS-8499) Refactor BlockInfo class hierarchy
 with static helper class
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639583#comment-14639583 ] 

Zhe Zhang commented on HDFS-8499:
---------------------------------

[~szetszwo] [~jingzhao] I wonder if you've had a chance to look at the above comment?

I went through the 2 UC classes again and confirmed my previous understanding on their differences. The UC feature class will look the same as trunk's {{BlockInfoUC}} except for the following lines (and changing {{replicas}} to directly use array, which we perhaps should do anyway for memory efficiency).

{code}
class UnderConstructionBlockStorageOp {
  public void setExpectedLocations(BlockInfo b, DatanodeStorageInfo[] targets) {
    ...
    Block blk = b.isStriped() ?
      new Block(this.getBlockId() + i, 0, this.getGenerationStamp()) : b;
    replicas[i] = new ReplicaUnderConstruction(blk, targets[i],
      ReplicaState.RBW);
    ...
  }

  public void addReplicaIfNotPresent(BlockInfo b, DatanodeStorageInfo storage,
        Block reportedBlock, ReplicaState rState) {
    ...
    if (expected == storage) {
      if (b.isStriped) {
        replicas[i].setBlockId(reportedBlock.getBlockId());
      }
    }
    ...
  }
}
{code}

After we convert contiguous/striped/UC operations into feature classes, we can basically tune the trade-off between code cleanness and overhead. E.g., if we don't want to maintain an additional pointer for UC feature we can use static methods in the feature class, and store UC fields ({{replicas}} etc.) in {{BIUCContiguous}} and {{BIUCStriped}}. All the code tuning will be contained in {{BlockInfo}} and its supporting classes. I'm more concerned about how other NN modules use {{BlockInfo}}. We are already creating a multi-inheritance and it will likely become more complex. IMHO, relying on type reflection and casting has the risk of causing us tricky bugs.

Please let me know if the above makes sense to you. I'm also happy to discuss alternative ways to resolve the {{BlockInfo}} gap between trunk and EC branch.

> Refactor BlockInfo class hierarchy with static helper class
> -----------------------------------------------------------
>
>                 Key: HDFS-8499
>                 URL: https://issues.apache.org/jira/browse/HDFS-8499
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 2.7.0
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>             Fix For: 2.8.0
>
>         Attachments: HDFS-8499.00.patch, HDFS-8499.01.patch, HDFS-8499.02.patch, HDFS-8499.03.patch, HDFS-8499.04.patch, HDFS-8499.05.patch, HDFS-8499.06.patch, HDFS-8499.07.patch, HDFS-8499.UCFeature.patch, HDFS-bistriped.patch
>
>
> In HDFS-7285 branch, the {{BlockInfoUnderConstruction}} interface provides a common abstraction for striped and contiguous UC blocks. This JIRA aims to merge it to trunk.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)