hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Surendra Singh Lilhore (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11726) [SPS] : StoragePolicySatisfier should not select same storage type as source and destination in same datanode.
Date Sun, 28 May 2017 13:30:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027821#comment-16027821

Surendra Singh Lilhore commented on HDFS-11726:

I got the root cause for this.

SPS first give the preference to the local DN. If expected storage type is not exist in source
dn then it will select remote DN. We missed one check in StoragePolicySatisfier#findSourceAndTargetToMove().
If replica with expected storage type already exist in source DN then we should skip that
For example:

Expected storage is : ARCHIVE
Source DN with storage : DN1(ARCHIVE), DN2(DISK).

We should not select DN1 for block movement because here already replica is exist in ARCHIVE.

> [SPS] : StoragePolicySatisfier should not select same storage type as source and destination
in same datanode.
> --------------------------------------------------------------------------------------------------------------
>                 Key: HDFS-11726
>                 URL: https://issues.apache.org/jira/browse/HDFS-11726
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>    Affects Versions: HDFS-10285
>            Reporter: Surendra Singh Lilhore
>            Assignee: Surendra Singh Lilhore
> {code}
> 2017-04-30 16:12:28,569 [BlockMoverTask-0] INFO  datanode.StoragePolicySatisfyWorker
(Worker.java:moveBlock(248)) - Start moving block:blk_1073741826_1002 from src:
to destin: to satisfy storageType, sourceStoragetype:ARCHIVE and destinStoragetype:ARCHIVE
> {code}
> {code}
> 2017-04-30 16:12:28,571 [DataXceiver for client / [Replacing block BP-1409501412-
from 6c7aa66e-a778-43d5-89f6-053d5f6b35bc]] INFO  datanode.DataNode (DataXceiver.java:replaceBlock(1202))
- opReplaceBlock BP-1409501412- received exception
org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica FinalizedReplica,
blk_1073741826_1002, FINALIZED
>   getNumBytes()     = 1024
>   getBytesOnDisk()  = 1024
>   getVisibleLength()= 1024
>   getVolume()       = /home/sachin/software/hadoop/HDFS-10285/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data7
>   getBlockURI()     = file:/home/sachin/software/hadoop/HDFS-10285/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data7/current/BP-1409501412-
already exists on storage ARCHIVE
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message