hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lei (Eddy) Xu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-6774) Make FsDataset and DataStore support removing volumes.
Date Sun, 17 Aug 2014 06:58:18 GMT

     [ https://issues.apache.org/jira/browse/HDFS-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Lei (Eddy) Xu updated HDFS-6774:

    Attachment: HDFS-6774.002.patch

Hi, [~atm] Thanks for the review. 

bq. The change in BlockPoolSlice - was that just a separate bug? Or why was that necessary?

It is indeed a separate bug. Should I open a new JIRA for this?

bq. I see the code where we remove the replica info from the replica map, but do we not also
need to do something similar in the event that the replica is currently referenced in the
BlockScanner or DirectoryScanner data structures? It could be that we don't, but I wanted
to check with you to see if you've considered this case.

Thanks for finding this bug. {{BlockScanner}} needs to remove the blocks and the corresponding
patch is attached. 

I think that {{DirectoryScanner}} does not need to change, since it uses {{FsDatasetImpl}}
as input source for each scan and performs {{DirectoryScanner#isValid()}}, which verifies
whether the volume is still available, when generate disk reports. This logic is the same
as handling disk failures. Therefore, {{DirectoryScanner}}'s consistency can be provided by
{{FsDatasetImpl}}. What is your opinion about this?

> Make FsDataset and DataStore support removing volumes.
> ------------------------------------------------------
>                 Key: HDFS-6774
>                 URL: https://issues.apache.org/jira/browse/HDFS-6774
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: datanode
>    Affects Versions: 2.4.1
>            Reporter: Lei (Eddy) Xu
>            Assignee: Lei (Eddy) Xu
>         Attachments: HDFS-6774.000.patch, HDFS-6774.001.patch, HDFS-6774.002.patch
> Managing volumes on DataNode includes decommissioning an active volume without restarting
> This task adds support to remove volumes from {{DataStorage}} and {{BlockPoolSliceStorage}}

This message was sent by Atlassian JIRA

View raw message