hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaobing Zhou (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11194) Maintain aggregated peer performance metrics on NameNode
Date Tue, 10 Jan 2017 01:38:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813493#comment-15813493
] 

Xiaobing Zhou edited comment on HDFS-11194 at 1/10/17 1:38 AM:
---------------------------------------------------------------

Thank you [~arpitagarwal] for the patch.
I've some comments.
# RollingAverages#getStats is using non-rolling mean states, it should use rolling ones instead.
See RollingAverages#snapshot for calculation of rolling averages.
# DFS_METRICS_ROLLING_AVERAGES_WINDOW_SIZE_DEFAULT should be changed to the same naming with
DFS_METRICS_ROLLING_AVERAGE_WINDOW_LENGTH_KEY
# These parameters can be changed to be configurable.
SlowNodeDetector#minOutlierDetectionPeers
DataNodePeerMetrics#LOW_THRESHOLD_MS
SlowPeerTracker#MAX_NODES_TO_REPORT
# In BlockReceiver#receivePacket, after trackSendPacketToLastNodeInPipeline(duration); It
may need to change a bit, e.g. if (duration > DataNodePeerMetrics#LOW_THRESHOLD_MS). or
remove the warning msg at all.


was (Author: xiaobingo):
Thank you [~arpitagarwal] for the patch.
I've some comments.
# RollingAverages#getStats is using non-rolling mean states, it should use rolling ones instead.
See RollingAverages#snapshot for calculation of rolling averages.

# DFS_METRICS_ROLLING_AVERAGES_WINDOW_SIZE_DEFAULT should be changed to the same naming with
DFS_METRICS_ROLLING_AVERAGE_WINDOW_LENGTH_KEY

# These parameters can be changed to be configurable.
SlowNodeDetector#minOutlierDetectionPeers
DataNodePeerMetrics#LOW_THRESHOLD_MS
SlowPeerTracker#MAX_NODES_TO_REPORT

# In BlockReceiver#receivePacket, after trackSendPacketToLastNodeInPipeline(duration); It
may need to change a bit, e.g. if (duration > DataNodePeerMetrics#LOW_THRESHOLD_MS). or
remove the warning msg at all.

> Maintain aggregated peer performance metrics on NameNode
> --------------------------------------------------------
>
>                 Key: HDFS-11194
>                 URL: https://issues.apache.org/jira/browse/HDFS-11194
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.8.0
>            Reporter: Xiaobing Zhou
>            Assignee: Arpit Agarwal
>         Attachments: HDFS-11194-03-04.delta, HDFS-11194.01.patch, HDFS-11194.02.patch,
HDFS-11194.03.patch, HDFS-11194.04.patch
>
>
> The metrics collected in HDFS-10917 should be reported to and aggregated on NameNode
as part of heart beat messages. This will make is easy to expose it through JMX to users who
are interested in them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message