hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9469) DiskBalancer : Add Planner
Date Wed, 23 Dec 2015 01:52:46 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15069021#comment-15069021

Arpit Agarwal commented on HDFS-9469:

Hi [~anu], my comments/questions. I apologize if some of this overlaps with the discussion
you had with [~szetszwo] earlier.

# DiskBalancerCluster.java:265: Will this be called by the executor?
# DiskBalancerCluster.java:346: These threads are used only for computing plans correct? It
looks like computing a plan for a node should be quite fast once we have collected the cluster
information? Perhaps we can avoid making it multi-threaded for now.
# GreedyPlanner.java:103: We remove the volumes from the set here and add them back in applyStep
below. Is that necessary, looks like we can skip both steps.
# GreedyPlanner.java:105: I think I need to convince myself this condition can be hit in practice
since we just recomputed idealUsed at the end of removeSkipVolumes.
# GreedyPlanner.java:116: Same question as above.
# GreedyPlanner.java:231: Can we use MoveStep#toString instead?
# DiskBalancerTestUtil.java:36: Curious why the switch from base-2 to base-10 units as much
as I dislike base-10 units.

Comments also posted at https://github.com/arp7/hadoop/commit/06caccf07da421e6e5525ffa7099a9e06d0a88ad

> DiskBalancer : Add Planner 
> ---------------------------
>                 Key: HDFS-9469
>                 URL: https://issues.apache.org/jira/browse/HDFS-9469
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>    Affects Versions: 2.8.0
>            Reporter: Anu Engineer
>            Assignee: Anu Engineer
>         Attachments: HDFS-9469-HDFS-1312.001.patch, HDFS-9469-HDFS-1312.002.patch, HDFS-9469-HDFS-1312.003.patch
> Disk Balancer reads the cluster data and then creates a plan for the data moves based
on the snap-shot of the data read from the nodes. This plan is later submitted to data nodes
for execution. 

This message was sent by Atlassian JIRA

View raw message