hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-14502) Confusion/name conflict between NameNodeActivity#BlockReportNumOps and RpcDetailedActivity#BlockReportNumOps
Date Wed, 21 Jun 2017 23:25:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-14502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhe Zhang updated HADOOP-14502:
-------------------------------
    Hadoop Flags: Incompatible change,Reviewed  (was: Incompatible change)

Thanks Erik! +1 on v2 patch as well. Tested with {{MiniHadoopClusterManager}} and it shows
desired behavior.
{code}
  }, {
    "name" : "Hadoop:service=NameNode,name=NameNodeActivity",
    "modelerType" : "NameNodeActivity",
    "tag.ProcessName" : "NameNode",
    "tag.SessionId" : null,
    "tag.Context" : "dfs",
    "tag.Hostname" : "zezhang-mn1",
    "CreateFileOps" : 2,
    "FilesCreated" : 12,
    "FilesAppended" : 0,
    "GetBlockLocations" : 0,
    "FilesRenamed" : 0,
    "FilesTruncated" : 0,
    "GetListingOps" : 1,
    "DeleteFileOps" : 0,
    "FilesDeleted" : 0,
    "FileInfoOps" : 6,
    "AddBlockOps" : 2,
    "GetAdditionalDatanodeOps" : 0,
    "CreateSymlinkOps" : 0,
    "GetLinkTargetOps" : 0,
    "FilesInGetListingOps" : 0,
    "AllowSnapshotOps" : 0,
    "DisallowSnapshotOps" : 0,
    "CreateSnapshotOps" : 0,
    "DeleteSnapshotOps" : 0,
    "RenameSnapshotOps" : 0,
    "ListSnapshottableDirOps" : 0,
    "SnapshotDiffReportOps" : 0,
    "BlockReceivedAndDeletedOps" : 2,
    "BlockOpsQueued" : 1,
    "BlockOpsBatched" : 0,
    "TransactionsNumOps" : 24,
    "TransactionsAvgTime" : 1.7083333333333333,
    "SyncsNumOps" : 14,
    "SyncsAvgTime" : 0.2857142857142857,
    "TransactionsBatchedInSync" : 10,
    "StorageBlockReportNumOps" : 2,
    "StorageBlockReportAvgTime" : 3.5,
    "CacheReportNumOps" : 0,
    "CacheReportAvgTime" : 0.0,
    "GenerateEDEKTimeNumOps" : 0,
    "GenerateEDEKTimeAvgTime" : 0.0,
    "WarmUpEDEKTimeNumOps" : 0,
    "WarmUpEDEKTimeAvgTime" : 0.0,
    "ResourceCheckTimeNumOps" : 8,
    "ResourceCheckTimeAvgTime" : 0.0,
    "SafeModeTime" : 1,
    "FsImageLoadTime" : 76,
    "GetEditNumOps" : 0,
    "GetEditAvgTime" : 0.0,
    "GetImageNumOps" : 0,
    "GetImageAvgTime" : 0.0,
    "PutImageNumOps" : 0,
    "PutImageAvgTime" : 0.0,
    "TotalFileOps" : 11
  },
{code}

I'm committing to trunk soon. Let's write a short release note?

> Confusion/name conflict between NameNodeActivity#BlockReportNumOps and RpcDetailedActivity#BlockReportNumOps
> ------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-14502
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14502
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: metrics
>            Reporter: Erik Krogen
>            Assignee: Erik Krogen
>            Priority: Minor
>              Labels: Incompatible
>         Attachments: HADOOP-14502.000.patch, HADOOP-14502.001.patch, HADOOP-14502.002.patch
>
>
> Currently the {{BlockReport(NumOps|AvgTime)}} metrics emitted under the {{RpcDetailedActivity}}
context and those emitted under the {{NameNodeActivity}} context are actually reporting different
things despite having the same name. {{NameNodeActivity}} reports the count/time of _per storage_
block reports, whereas {{RpcDetailedActivity}} reports the count/time of _per datanode_ block
reports. This makes for a confusing experience with two metrics having the same name reporting
different values. 
> We already have the {{StorageBlockReportsOps}} metric under {{NameNodeActivity}}. Can
we make {{StorageBlockReport}} a {{MutableRate}} metric and remove {{NameNodeActivity#BlockReport}}
metric? Open to other suggestions about how to address this as well. The 3.0 release seems
a good time to make this incompatible change.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message