hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11545) Propagate DataNode's slow disks info to the NameNode via Heartbeat
Date Sun, 19 Mar 2017 20:43:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15931954#comment-15931954

Arpit Agarwal commented on HDFS-11545:

Thanks [~hanishakoneru]. The patch looks great. Minor feedback:
# The following comment in DatanodeProtocol.proto looks unnecessary since SlowDiskReportProto
does not include the DataNodeId.
 * DataNode. (DataNodeId must be transmitted as a string for protocol
 * compability with earlier versions of Hadoop).
# NUM_DISK_OPS can be initialized to {{DiskOp.values().length()}} instead of hard-coding 3.
# Extra space in {{return  slowDiskInfoProtos}}.

We should add a few unit tests:
# Test for PBHelper serialization/deserialization. See TestPBHelper#testSlowPeerInfoPBHelper.
# Ensure that the datanode sends out a slow disk report when one is due, and it includes the
contents of {{dn.getDiskMetrics().getDiskOutliersStats()}}.

> Propagate DataNode's slow disks info to the NameNode via Heartbeat
> ------------------------------------------------------------------
>                 Key: HDFS-11545
>                 URL: https://issues.apache.org/jira/browse/HDFS-11545
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Hanisha Koneru
>            Assignee: Hanisha Koneru
>         Attachments: HDFS-11545.001.patch
> DataNode detects the outliers (slow disks) among all its disk. This information can be
propagated to the NameNode so that the NameNode gets disk information from all Datanodes.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message