hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9173) Erasure Coding: Lease recovery for striped file
Date Thu, 15 Oct 2015 23:37:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959859#comment-14959859
] 

Zhe Zhang commented on HDFS-9173:
---------------------------------

Thanks Walter for the updates. Some additional comments:

# In the current patch {{StripedRecoveryTask}} only shares a few lines of trivial code with
{{RecoveryTask}}. It's a little odd to subclass it.
# Fundamentally the 2 {{RecoveryTask}} types do share a lot of logics. They both go through
steps 1, 2, 5 as described in the Javadoc. So here's an alternative way to structure the 2
classes:
{code}
public class RecoveryTaskContiguous {
  protected void recover() {
    // Step 1.1: callInitReplicaRecovery and get all blocks lengths from DataNodes, generate
list of BlockRecord
    // Step 1.2: check if there's any FINALIZED replica
  }
  void syncBlockFinalized(List<BlockRecord> syncList) {
  }
  void syncBlockUnfinalized(List<BlockRecord> syncList) {
  }
}

public class RecoveryTaskStriped {
  protected void recover() {
    // Step 1.1: callInitReplicaRecovery and get all blocks lengths from DataNodes, generate
list of BlockRecord
  }
  void syncBlockUnfinalized(List<BlockRecord> syncList) {
  }
}
{code}
Step 1.1 is identical in both classes so we can use a shared static method to do it. The logic
of synchronizing striped internal blocks is very similar to handling {{RBW}} and {{RWR}} replicas.
We can use a shared {{syncBlockUnfinalized}} method. This suggested consolidation can be done
separately.
# In {{StripedRecoveryTask#recover}} we are calling {{callInitReplicaRecovery}} twice. Is
the second call necessary?
# {{StripedRecoveryTask#ecPolicy}} is unnecessary.

Nits:
# We can take this chance to fix the {{if}} statements without parenthesis
# Can also update ""Convenience"" / "convenient" to "helper" / "util" when describing classes
and methods
# Steps 3 and 4 in the Javadoc should be marked as TODO or removed for now

> Erasure Coding: Lease recovery for striped file
> -----------------------------------------------
>
>                 Key: HDFS-9173
>                 URL: https://issues.apache.org/jira/browse/HDFS-9173
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HDFS-9173.00.wip.patch, HDFS-9173.01.patch, HDFS-9173.02.step125.patch,
HDFS-9173.03.patch, HDFS-9173.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message