hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6997) Archival Storage: add more tests for data migration and replicaion
Date Thu, 09 Apr 2015 22:58:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14488448#comment-14488448
] 

Andrew Wang commented on HDFS-6997:
-----------------------------------

This JIRA includes a pretty major bug fix for per-storage space calculation in BlockPlacementPolicyDefault#isGoodTarget.
We just hit a situation where an entire cluster became unavailable for write because the free
space per storage calculation was using the per-node # scheduled blocks. So, even though the
individual disks had plenty of space left, the NN thought the storages were all full. I consider
this a blocker bug, and it exists in any release with HSM but without archival storage.

Since this JIRA was marked as "Minor" and the summary simply says "add more tests", I was
rather surprised to see it show up in git blame. Next time, let's revise the JIRA fields as
well as also including relevant bug fixes in trunk and branch-2.

> Archival Storage: add more tests for data migration and replicaion
> ------------------------------------------------------------------
>
>                 Key: HDFS-6997
>                 URL: https://issues.apache.org/jira/browse/HDFS-6997
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover, namenode
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>            Priority: Minor
>             Fix For: Archival Storage (HDFS-6584)
>
>         Attachments: h6997_20140905.patch, h6997_20140905b.patch, h6997_20140907.patch
>
>
> This JIRA is to add more tests to check if the data migration tool could move the replicas
correctly and if the replication monitor could replicate block correctly when storage policy
is considered.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message