hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Krogen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14502) Confusion/name conflict between NameNodeActivity#BlockReportNumOps and RpcDetailedActivity#BlockReportNumOps
Date Thu, 08 Jun 2017 16:06:18 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Erik Krogen updated HADOOP-14502:
---------------------------------
    Attachment: HADOOP-14502.000.patch

Attaching initial patch. I updated the quantile metrics for BlockReports as well since they
also reflect individual storages.

Old metrics output:
{code}
 }, {
    "name" : "Hadoop:service=NameNode,name=RpcDetailedActivityForPort58010",
    "BlockReportNumOps" : 2,
    "BlockReportAvgTime" : 42.0,
...
  }, {
    "name" : "Hadoop:service=NameNode,name=NameNodeActivity",
    "BlockReport60sNumOps" : 0,
    "BlockReport60s50thPercentileLatency" : 0,
    "BlockReport60s75thPercentileLatency" : 0,
    "BlockReport60s90thPercentileLatency" : 0,
    "BlockReport60s95thPercentileLatency" : 0,
    "BlockReport60s99thPercentileLatency" : 0,
    "StorageBlockReportOps" : 4,
    "BlockReportOps" : 4,
    "BlockReportAvgTime" : 5.5,
...
  }, {
{code}

New metrics output:
{code}
 }, {
    "name" : "Hadoop:service=NameNode,name=RpcDetailedActivityForPort58010",
    "BlockReportNumOps" : 2,
    "BlockReportAvgTime" : 42.0,
...
  }, {
    "name" : "Hadoop:service=NameNode,name=NameNodeActivity",
    "StorageBlockReport60sNumOps" : 0,
    "StorageBlockReport60s50thPercentileLatency" : 0,
    "StorageBlockReport60s75thPercentileLatency" : 0,
    "StorageBlockReport60s90thPercentileLatency" : 0,
    "StorageBlockReport60s95thPercentileLatency" : 0,
    "StorageBlockReport60s99thPercentileLatency" : 0,
    "StorageBlockReportNumOps" : 4,
    "StorageBlockReportAvgTime" : 5.5,
...
  }, {
{code}

I also would like to point out this is consistent with the DataNode BlockReport metric:
{code}
}, {
    "name" : "Hadoop:service=DataNode,name=DataNodeActivity-127.0.0.1-58011",
    "BlockReportsNumOps" : 2,
    "BlockReportsAvgTime" : 65.0,
...
  }, {
{code}

> Confusion/name conflict between NameNodeActivity#BlockReportNumOps and RpcDetailedActivity#BlockReportNumOps
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14502
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14502
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>            Priority: Minor
>         Attachments: HADOOP-14502.000.patch
>
>
> Currently the {{BlockReport(NumOps|AvgTime)}} metrics emitted under the {{RpcDetailedActivity}}
context and those emitted under the {{NameNodeActivity}} context are actually reporting different
things despite having the same name. {{NameNodeActivity}} reports the count/time of _per storage_
block reports, whereas {{RpcDetailedActivity}} reports the count/time of _per datanode_ block
reports. This makes for a confusing experience with two metrics having the same name reporting
different values. 
> We already have the {{StorageBlockReportsOps}} metric under {{NameNodeActivity}}. Can
we make {{StorageBlockReport}} a {{MutableRate}} metric and remove {{NameNodeActivity#BlockReport}}
metric? Open to other suggestions about how to address this as well. The 3.0 release seems
a good time to make this incompatible change.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message