hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-7411) Refactor decommissioning logic into DecommissionManager
Date Sat, 06 Dec 2014 02:17:13 GMT

     [ https://issues.apache.org/jira/browse/HDFS-7411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Wang updated HDFS-7411:
    Attachment: hdfs-7411.003.patch

I spent some more time and did a much bigger rework of DecommissionManager. The more I got
into it, the more I found, so this is essentially a rewrite.

The highlights:

* Moved almost all the decom logic into DecomManager, significant reworks, including the above
* Introduced the idea of an "insufficiently replicated" block. A block can be under-replicated
but not block decommissioning.
* Monitor scans now iterate for a number of blocks, rather than a number of nodes. This should
be much smoother. The configuration becomes a little unfortunate, but this scheme preserves
backwards compatibility.
* Switched a few things to SLF4J
* Added lots of comments
* Cleaned up decom tests a good bit
* Fixed a bug in ChunkedArrayList, since its size does not respect iterator.remove()

> Refactor decommissioning logic into DecommissionManager
> -------------------------------------------------------
>                 Key: HDFS-7411
>                 URL: https://issues.apache.org/jira/browse/HDFS-7411
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.5.1
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>         Attachments: hdfs-7411.001.patch, hdfs-7411.002.patch, hdfs-7411.003.patch
> Would be nice to split out decommission logic from DatanodeManager to DecommissionManager.

This message was sent by Atlassian JIRA

View raw message