hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaobing Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-9462) DiskBalancer: Add Query Command
Date Tue, 24 Nov 2015 22:52:11 GMT

     [ https://issues.apache.org/jira/browse/HDFS-9462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Xiaobing Zhou updated HDFS-9462:
--------------------------------
    Description: 
This is to propose being able to query all the nodes that we send various plans to. In order
to do the query, query command will talk to all data nodes through cluster interface in and
compare the hash tag that we get back to make sure that is the plan that we are interested
in and print out the results.

We also want to support the ability to print out the diff of what happened when a run is complete.
Assuming  the state of the cluster is saved to file before.json. There should be two kinds
of diffs.

1. Overall what happened in the cluster vs. before.json -- just a summary 
2. for a specific node -- just like report command we should be able to pass in a node and
as see the changes against the before.json


  was:
It's quite helpful to do:
1) report node information for the top X of DataNodes that will benefit from running disk
balancer
2) report volume level information for any specific DataNode. 

This is done by:
1) reading the cluster info, sorting the DiskbalancerNodes by their NodeDataDensity and printing
out their corresponding information.
2) reading the cluster info, and print out volume level information for that DataNode requested.


> DiskBalancer: Add Query Command
> -------------------------------
>
>                 Key: HDFS-9462
>                 URL: https://issues.apache.org/jira/browse/HDFS-9462
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>    Affects Versions: 2.8.0
>            Reporter: Xiaobing Zhou
>            Assignee: Xiaobing Zhou
>
> This is to propose being able to query all the nodes that we send various plans to. In
order to do the query, query command will talk to all data nodes through cluster interface
in and compare the hash tag that we get back to make sure that is the plan that we are interested
in and print out the results.
> We also want to support the ability to print out the diff of what happened when a run
is complete. Assuming  the state of the cluster is saved to file before.json. There should
be two kinds of diffs.
> 1. Overall what happened in the cluster vs. before.json -- just a summary 
> 2. for a specific node -- just like report command we should be able to pass in a node
and as see the changes against the before.json



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message