hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8499) Refactor BlockInfo class hierarchy with static helper class
Date Thu, 23 Jul 2015 21:57:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14639583#comment-14639583
] 

Zhe Zhang commented on HDFS-8499:
---------------------------------

[~szetszwo] [~jingzhao] I wonder if you've had a chance to look at the above comment?

I went through the 2 UC classes again and confirmed my previous understanding on their differences.
The UC feature class will look the same as trunk's {{BlockInfoUC}} except for the following
lines (and changing {{replicas}} to directly use array, which we perhaps should do anyway
for memory efficiency).

{code}
class UnderConstructionBlockStorageOp {
  public void setExpectedLocations(BlockInfo b, DatanodeStorageInfo[] targets) {
    ...
    Block blk = b.isStriped() ?
      new Block(this.getBlockId() + i, 0, this.getGenerationStamp()) : b;
    replicas[i] = new ReplicaUnderConstruction(blk, targets[i],
      ReplicaState.RBW);
    ...
  }

  public void addReplicaIfNotPresent(BlockInfo b, DatanodeStorageInfo storage,
        Block reportedBlock, ReplicaState rState) {
    ...
    if (expected == storage) {
      if (b.isStriped) {
        replicas[i].setBlockId(reportedBlock.getBlockId());
      }
    }
    ...
  }
}
{code}

After we convert contiguous/striped/UC operations into feature classes, we can basically tune
the trade-off between code cleanness and overhead. E.g., if we don't want to maintain an additional
pointer for UC feature we can use static methods in the feature class, and store UC fields
({{replicas}} etc.) in {{BIUCContiguous}} and {{BIUCStriped}}. All the code tuning will be
contained in {{BlockInfo}} and its supporting classes. I'm more concerned about how other
NN modules use {{BlockInfo}}. We are already creating a multi-inheritance and it will likely
become more complex. IMHO, relying on type reflection and casting has the risk of causing
us tricky bugs.

Please let me know if the above makes sense to you. I'm also happy to discuss alternative
ways to resolve the {{BlockInfo}} gap between trunk and EC branch.

> Refactor BlockInfo class hierarchy with static helper class
> -----------------------------------------------------------
>
>                 Key: HDFS-8499
>                 URL: https://issues.apache.org/jira/browse/HDFS-8499
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: 2.7.0
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>             Fix For: 2.8.0
>
>         Attachments: HDFS-8499.00.patch, HDFS-8499.01.patch, HDFS-8499.02.patch, HDFS-8499.03.patch,
HDFS-8499.04.patch, HDFS-8499.05.patch, HDFS-8499.06.patch, HDFS-8499.07.patch, HDFS-8499.UCFeature.patch,
HDFS-bistriped.patch
>
>
> In HDFS-7285 branch, the {{BlockInfoUnderConstruction}} interface provides a common abstraction
for striped and contiguous UC blocks. This JIRA aims to merge it to trunk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message