Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 18D80200C04 for ; Tue, 10 Jan 2017 02:39:00 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 1781E160B49; Tue, 10 Jan 2017 01:39:00 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 62E54160B3E for ; Tue, 10 Jan 2017 02:38:59 +0100 (CET) Received: (qmail 30407 invoked by uid 500); 10 Jan 2017 01:38:58 -0000 Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list hdfs-issues@hadoop.apache.org Received: (qmail 30376 invoked by uid 99); 10 Jan 2017 01:38:58 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Jan 2017 01:38:58 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 6D8792C0453 for ; Tue, 10 Jan 2017 01:38:58 +0000 (UTC) Date: Tue, 10 Jan 2017 01:38:58 +0000 (UTC) From: "Xiaobing Zhou (JIRA)" To: hdfs-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (HDFS-11194) Maintain aggregated peer performance metrics on NameNode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 10 Jan 2017 01:39:00 -0000 [ https://issues.apache.org/jira/browse/HDFS-11194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15813493#comment-15813493 ] Xiaobing Zhou edited comment on HDFS-11194 at 1/10/17 1:38 AM: --------------------------------------------------------------- Thank you [~arpitagarwal] for the patch. I've some comments. # RollingAverages#getStats is using non-rolling mean states, it should use rolling ones instead. See RollingAverages#snapshot for calculation of rolling averages. # DFS_METRICS_ROLLING_AVERAGES_WINDOW_SIZE_DEFAULT should be changed to the same naming with DFS_METRICS_ROLLING_AVERAGE_WINDOW_LENGTH_KEY # These parameters can be changed to be configurable. SlowNodeDetector#minOutlierDetectionPeers DataNodePeerMetrics#LOW_THRESHOLD_MS SlowPeerTracker#MAX_NODES_TO_REPORT # In BlockReceiver#receivePacket, after trackSendPacketToLastNodeInPipeline(duration); It may need to change a bit, e.g. if (duration > DataNodePeerMetrics#LOW_THRESHOLD_MS). or remove the warning msg at all. was (Author: xiaobingo): Thank you [~arpitagarwal] for the patch. I've some comments. # RollingAverages#getStats is using non-rolling mean states, it should use rolling ones instead. See RollingAverages#snapshot for calculation of rolling averages. # DFS_METRICS_ROLLING_AVERAGES_WINDOW_SIZE_DEFAULT should be changed to the same naming with DFS_METRICS_ROLLING_AVERAGE_WINDOW_LENGTH_KEY # These parameters can be changed to be configurable. SlowNodeDetector#minOutlierDetectionPeers DataNodePeerMetrics#LOW_THRESHOLD_MS SlowPeerTracker#MAX_NODES_TO_REPORT # In BlockReceiver#receivePacket, after trackSendPacketToLastNodeInPipeline(duration); It may need to change a bit, e.g. if (duration > DataNodePeerMetrics#LOW_THRESHOLD_MS). or remove the warning msg at all. > Maintain aggregated peer performance metrics on NameNode > -------------------------------------------------------- > > Key: HDFS-11194 > URL: https://issues.apache.org/jira/browse/HDFS-11194 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode > Affects Versions: 2.8.0 > Reporter: Xiaobing Zhou > Assignee: Arpit Agarwal > Attachments: HDFS-11194-03-04.delta, HDFS-11194.01.patch, HDFS-11194.02.patch, HDFS-11194.03.patch, HDFS-11194.04.patch > > > The metrics collected in HDFS-10917 should be reported to and aggregated on NameNode as part of heart beat messages. This will make is easy to expose it through JMX to users who are interested in them. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org