Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 8 Jun 2017 16:06:18 +0000 (UTC)
From: "Erik Krogen (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: <JIRA.13078072.1496859801000.29692.1496937978499@Atlassian.JIRA>
In-Reply-To: <JIRA.13078072.1496859801000@Atlassian.JIRA>
References: <JIRA.13078072.1496859801000@Atlassian.JIRA> <JIRA.13078072.1496859801427@jira-lw-us.apache.org>
Subject: [jira] [Updated] (HADOOP-14502) Confusion/name conflict between
 NameNodeActivity#BlockReportNumOps and
 RpcDetailedActivity#BlockReportNumOps
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 08 Jun 2017 16:06:29 -0000


     [ https://issues.apache.org/jira/browse/HADOOP-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erik Krogen updated HADOOP-14502:
---------------------------------
    Attachment: HADOOP-14502.000.patch

Attaching initial patch. I updated the quantile metrics for BlockReports as well since they also reflect individual storages.

Old metrics output:
{code}
 }, {
    "name" : "Hadoop:service=NameNode,name=RpcDetailedActivityForPort58010",
    "BlockReportNumOps" : 2,
    "BlockReportAvgTime" : 42.0,
...
  }, {
    "name" : "Hadoop:service=NameNode,name=NameNodeActivity",
    "BlockReport60sNumOps" : 0,
    "BlockReport60s50thPercentileLatency" : 0,
    "BlockReport60s75thPercentileLatency" : 0,
    "BlockReport60s90thPercentileLatency" : 0,
    "BlockReport60s95thPercentileLatency" : 0,
    "BlockReport60s99thPercentileLatency" : 0,
    "StorageBlockReportOps" : 4,
    "BlockReportOps" : 4,
    "BlockReportAvgTime" : 5.5,
...
  }, {
{code}

New metrics output:
{code}
 }, {
    "name" : "Hadoop:service=NameNode,name=RpcDetailedActivityForPort58010",
    "BlockReportNumOps" : 2,
    "BlockReportAvgTime" : 42.0,
...
  }, {
    "name" : "Hadoop:service=NameNode,name=NameNodeActivity",
    "StorageBlockReport60sNumOps" : 0,
    "StorageBlockReport60s50thPercentileLatency" : 0,
    "StorageBlockReport60s75thPercentileLatency" : 0,
    "StorageBlockReport60s90thPercentileLatency" : 0,
    "StorageBlockReport60s95thPercentileLatency" : 0,
    "StorageBlockReport60s99thPercentileLatency" : 0,
    "StorageBlockReportNumOps" : 4,
    "StorageBlockReportAvgTime" : 5.5,
...
  }, {
{code}

I also would like to point out this is consistent with the DataNode BlockReport metric:
{code}
}, {
    "name" : "Hadoop:service=DataNode,name=DataNodeActivity-127.0.0.1-58011",
    "BlockReportsNumOps" : 2,
    "BlockReportsAvgTime" : 65.0,
...
  }, {
{code}

> Confusion/name conflict between NameNodeActivity#BlockReportNumOps and RpcDetailedActivity#BlockReportNumOps
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14502
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14502
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>            Priority: Minor
>         Attachments: HADOOP-14502.000.patch
>
>
> Currently the {{BlockReport(NumOps|AvgTime)}} metrics emitted under the {{RpcDetailedActivity}} context and those emitted under the {{NameNodeActivity}} context are actually reporting different things despite having the same name. {{NameNodeActivity}} reports the count/time of _per storage_ block reports, whereas {{RpcDetailedActivity}} reports the count/time of _per datanode_ block reports. This makes for a confusing experience with two metrics having the same name reporting different values. 
> We already have the {{StorageBlockReportsOps}} metric under {{NameNodeActivity}}. Can we make {{StorageBlockReport}} a {{MutableRate}} metric and remove {{NameNodeActivity#BlockReport}} metric? Open to other suggestions about how to address this as well. The 3.0 release seems a good time to make this incompatible change.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org