hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11545) Propagate DataNode's slow disks info to the NameNode via Heartbeat
Date Sun, 19 Mar 2017 20:43:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931954#comment-15931954
] 

Arpit Agarwal commented on HDFS-11545:
--------------------------------------

Thanks [~hanishakoneru]. The patch looks great. Minor feedback:
# The following comment in DatanodeProtocol.proto looks unnecessary since SlowDiskReportProto
does not include the DataNodeId.
{code}
 * DataNode. (DataNodeId must be transmitted as a string for protocol
 * compability with earlier versions of Hadoop).
{code}
# NUM_DISK_OPS can be initialized to {{DiskOp.values().length()}} instead of hard-coding 3.
# Extra space in {{return  slowDiskInfoProtos}}.

We should add a few unit tests:
# Test for PBHelper serialization/deserialization. See TestPBHelper#testSlowPeerInfoPBHelper.
# Ensure that the datanode sends out a slow disk report when one is due, and it includes the
contents of {{dn.getDiskMetrics().getDiskOutliersStats()}}.

> Propagate DataNode's slow disks info to the NameNode via Heartbeat
> ------------------------------------------------------------------
>
>                 Key: HDFS-11545
>                 URL: https://issues.apache.org/jira/browse/HDFS-11545
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Hanisha Koneru
>            Assignee: Hanisha Koneru
>         Attachments: HDFS-11545.001.patch
>
>
> DataNode detects the outliers (slow disks) among all its disk. This information can be
propagated to the NameNode so that the NameNode gets disk information from all Datanodes.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message