hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9205) Do not schedule corrupt blocks for replication
Date Thu, 15 Oct 2015 10:15:07 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14958668#comment-14958668
] 

Hudson commented on HDFS-9205:
------------------------------

FAILURE: Integrated in Hadoop-trunk-Commit #8641 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8641/])
HDFS-9205. Do not schedule corrupt blocks for replication.  (szetszwo) (szetszwo: rev 5411dc559d5f73e4153e76fdff94a26869c17a37)
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestUnderReplicatedBlockQueues.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/NumberReplicas.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/UnderReplicatedBlocks.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java


> Do not schedule corrupt blocks for replication
> ----------------------------------------------
>
>                 Key: HDFS-9205
>                 URL: https://issues.apache.org/jira/browse/HDFS-9205
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>            Priority: Minor
>         Attachments: h9205_20151007.patch, h9205_20151007b.patch, h9205_20151008.patch,
h9205_20151009.patch, h9205_20151009b.patch, h9205_20151013.patch, h9205_20151015.patch
>
>
> Corrupted blocks by definition are blocks cannot be read. As a consequence, they cannot
be replicated.  In UnderReplicatedBlocks, there is a queue for QUEUE_WITH_CORRUPT_BLOCKS and
chooseUnderReplicatedBlocks may choose blocks from it.  It seems that scheduling corrupted
block for replication is wasting resource and potentially slow down replication for the higher
priority blocks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message