hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8688) replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack
Date Fri, 17 Jul 2015 03:37:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14630715#comment-14630715
] 

Walter Su commented on HDFS-8688:
---------------------------------

bq. There are several other places that call blockHasEnoughRacks directly only if there is
no pending replications for that block. Maybe we can do the same for addStoredBlock case in
addition to the current patch. In that way, we make sure blockHasEnoughRacks will be called
only when there is enough liveReplicas.
Agree.
If pendingReplication not null, we don't need check blockHasEnoughRacks(). We wait pendingReplication
to be finished, and we check blockHasEnoughRacks() again later.
That's the appropriate way.

---------
bq. There are several other places that call blockHasEnoughRacks directly only if there is
no pending replications for that block. Maybe we can do the same for addStoredBlock case in
addition to the current patch. In that way, we make sure blockHasEnoughRacks will be called
only when there is enough liveReplicas.

1. >...if there is pendingReplications, NN still calls blockHasEnoughRacks() in addStoredBlock.
Yes. Good catch!
2. >...we can do the same for addStoredBlock ...
No, we don't have to.
{code}
2703     if (!isNeededReplication(storedBlock, fileReplication, numCurrentReplica)) {
2704       neededReplications.remove(storedBlock, numCurrentReplica,
2705           num.decommissionedAndDecommissioning(), fileReplication);
2706     } else {
2707       updateNeededReplications(storedBlock, curReplicaDelta, 0);
2708     }
{code}

You must be confused why line 2703 uses {{numCurrentReplica}} but not live nodes. In fact
the logic is correct. Because line 2704 is neededReplications.remove(..). If the number of
racks already meet, it's natrual to remove from neededReplications.
line 2707 take effect only if in safemode.

In general, addStoredBlock will NOT add block to neededReplications. We don't have to change
anything about it.

> replace shouldCheckForEnoughRacks with hasClusterEverBeenMultiRack
> ------------------------------------------------------------------
>
>                 Key: HDFS-8688
>                 URL: https://issues.apache.org/jira/browse/HDFS-8688
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Walter Su
>            Assignee: Walter Su
>         Attachments: HDFS-8688.01.patch, HDFS-8688.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message