hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8147) Mover should not schedule two replicas to the same DN storage
Date Fri, 24 Apr 2015 11:37:45 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14510883#comment-14510883

Hudson commented on HDFS-8147:

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/])
HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode.  Contributed by
surendra singh lilhore (szetszwo: rev 416b84354ebcc6f6f23598a5a254e62eb606680d)
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestMover.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

> Mover should not schedule two replicas to the same DN storage
> -------------------------------------------------------------
>                 Key: HDFS-8147
>                 URL: https://issues.apache.org/jira/browse/HDFS-8147
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: balancer & mover
>    Affects Versions: 2.6.0
>            Reporter: surendra singh lilhore
>            Assignee: surendra singh lilhore
>             Fix For: 2.7.1
>         Attachments: HDFS-8147.patch, HDFS-8147_1.patch, HDFS-8147_2.patch, HDFS-8147_3.patch,
> *Scenario:*
> 1. Three DN cluster.  For DNs storage type is like this.
>         DN1 : DISK,ARCHIVE
>         DN2 : DISK
>         DN3 : DISK,ARCHIVE (All DNs are in same rack)
> 2. One file with two replicas (In DN1 and DN2)
> 3. Set file storage policy COLD
> 4. Now execute Mover.
> *Expected Result:* File blocks should move in DN1:ARCHIVE and DN3:ARCHIVE
> *Actual Result:* {{chooseTargetInSameNode()}} move D1:DISK block to D1:ARCHIVE, but in
next iteration {{chooseTarget()}} for same rake is selecting again DN1:ARCHIVE for target
where already same block exists.
> {{chooseTargetInSameNode()}} and {{chooseTarget()}} should not select the node as target
where already same replica exists.  The dispatcher will fail to move block as shown in the
log below.  Then, the Mover will try again in next iteration.
> *Logs*
> {code}
> 15/04/15 10:47:17 WARN balancer.Dispatcher: Failed to move blk_1073741852_1028 with size=11990
from to through Got error,
status message opReplaceBlock BP-1258709199-
received exception org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica
FinalizedReplica, blk_1073741852_1028, FINALIZED
> {code}

This message was sent by Atlassian JIRA

View raw message