hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manoj Govindassamy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9850) DiskBalancer : Explore removing references to FsVolumeSpi
Date Tue, 27 Sep 2016 19:06:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15527123#comment-15527123

Manoj Govindassamy commented on HDFS-9850:

Thanks for the comments.

* Old behavior:
## In {{DiskBalancerMover#copyBlocks()}}, this.dataset.moveBlockAcrossVolumes(block, dest)
fails with {{ClosedChannelException}} when it tries to  get the volume reference.
## Yes, queryStatus() gets the volume base path as it already has the FsVolumeSpi reference

* New behavior:
## Yes, the new fix sets the error message when the volume is not available. But, the fix
does not take care of currentResult state and it will still be in PLAN_UNDER_PROGRESS
## Yes, this is different from old behavior as queryStatus would return an error instead of
the stored paths.

Got your point for getQueryStatus(). Since FsVolumeSpi references are no more stored and only
StorageVolume UUIDs are available, its not possible to construct the path for a non-existing

ConcurrentHashMap<VolumePair, DiskBalancerWorkItem> workMap;

  public static class VolumePair {
    private final String sourceVolUuid;
    private final String destVolUuid;

Here are few other options on fixing queryWorkStatus:
# Instead of throwing error, may be we can report volume path as <volume missing> or
related error message whenever we find the volume removed
# Or, persist volume paths also along with UUID, so that query will always work as before

Your thoughts please ?

> DiskBalancer : Explore removing references to FsVolumeSpi 
> ----------------------------------------------------------
>                 Key: HDFS-9850
>                 URL: https://issues.apache.org/jira/browse/HDFS-9850
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>    Affects Versions: 3.0.0-alpha2
>            Reporter: Anu Engineer
>            Assignee: Manoj Govindassamy
>         Attachments: HDFS-9850.001.patch, HDFS-9850.002.patch, HDFS-9850.003.patch
> In HDFS-9671, [~arpitagarwal] commented that we should explore the possibility of removing
references to FsVolumeSpi at any point and only deal with storage ID. We are not sure if this
is possible, this JIRA is to explore if that can be done without issues.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message