hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8147) Mover should not select the DN storage as target where already same replica exists.
Date Fri, 17 Apr 2015 22:08:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14500782#comment-14500782
] 

Tsz Wo Nicholas Sze commented on HDFS-8147:
-------------------------------------------

I think the problem is that when the Mover schedule the moves DN1 ARCHIVE does not has the
block.  When dispatching the blocks, one of them is dispatched successfully but the other
will fail.

We already have block.isLocatedOn(target) in isGoodBlockCandidate(..).  So I think the patch
may not help.  Could you test it?

> Mover should not select the DN storage as target where already same replica exists.
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-8147
>                 URL: https://issues.apache.org/jira/browse/HDFS-8147
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: balancer & mover
>    Affects Versions: 2.6.0
>            Reporter: surendra singh lilhore
>            Assignee: surendra singh lilhore
>         Attachments: HDFS-8147.patch
>
>
> *Scenario:*
> 1. Three DN cluster.  For DNs storage type is like this.
>         DN1 : DISK,ARCHIVE
>         DN2 : DISK
>         DN3 : DISK,ARCHIVE (All DNs are in same rack)
> 2. One file with two replicas (In DN1 and DN2)
> 3. Set file storage policy COLD
> 4. Now execute Mover.
> *Expected Result:* File blocks should move in DN1:ARCHIVE and DN3:ARCHIVE
> *Actual Result:* {{chooseTargetInSameNode()}} move D1:DISK block to D1:ARCHIVE, but in
next iteration {{chooseTarget()}} for same rake is selecting again DN1:ARCHIVE for target
where already same block exists.
> {{chooseTargetInSameNode()}} and {{chooseTarget()}} should not select the node as target
where already same replica exists.
> *Logs*
> {code}
> 15/04/15 10:47:17 WARN balancer.Dispatcher: Failed to move blk_1073741852_1028 with size=11990
from 10.19.92.74:50010:DISK to 10.19.92.73:50010:ARCHIVE through 10.19.92.73:50010: Got error,
status message opReplaceBlock BP-1258709199-10.19.92.74-1428292615636:blk_1073741852_1028
received exception org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica
FinalizedReplica, blk_1073741852_1028, FINALIZED
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message