hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-11564) Ozone: SCM: Add Comparable Metric Support
Date Fri, 31 Mar 2017 21:38:41 GMT

     [ https://issues.apache.org/jira/browse/HDFS-11564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Anu Engineer updated HDFS-11564:
    Attachment: HDFS-11564-HDFS-7240.004.patch

[~xyao] Thanks for the review. I have updated the patch based on your comments.

bq. Line 150: Can you add javadoc for the purpose of this public method getResultSet()?

bq. Maybe we can consolidate chooseDatanodes() and getResultSet()? It seems the former consider
both numNodes and size and the later consider only numNodes.
I don't think we can.  The chooseDatanode filters out a set of nodes from the healthy
nodes that has enough free space to meet the required size.

Then getResultSet calls into the derived classes chooseNode function and once that is done,
it has to verify that we did get as many as requested nodes.

bq. Do we attempt to handle the case where we can't find enough ndoe with required capacity
different from SCMCommonPolicy?

bq.  Define separate class for Remaining/ScmUsed/Capacity is bit over-engineering. Is it possible
to just use LongMetric.

bq. SCMException.java -- Renames

> Ozone: SCM: Add Comparable Metric Support
> -----------------------------------------
>                 Key: HDFS-11564
>                 URL: https://issues.apache.org/jira/browse/HDFS-11564
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Anu Engineer
>         Attachments: HDFS-11564-HDFS-7240.001.patch, HDFS-11564-HDFS-7240.002.patch,
HDFS-11564-HDFS-7240.003.patch, HDFS-11564-HDFS-7240.004.patch
> While picking the datanode or the pool to place a container, SCM needs to be able to
compare pools and datanodes. This jira proposes to add a Comparable Metric interface from
which DatanodeCapacity and DatanodeIOLoad metrics can be derived. This will also convert SCM
Container Placement Capacity policy to use these metrics.
> Adding these metrics makes all placement policy operate on the basis of these metrics
and it makes it easy to simulate cluster imbalances as well as reason about code. All new
SCM metrics which have the property of total ordering should be derived from this interface.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message