hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From viraj...@apache.org
Subject [47/50] [abbrv] hadoop git commit: HDFS-13674. Improve documentation on Metrics. Contributed by Chao Sun.
Date Mon, 09 Jul 2018 18:26:39 GMT
HDFS-13674. Improve documentation on Metrics. Contributed by Chao Sun.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/7a68ac60
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/7a68ac60
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/7a68ac60

Branch: refs/heads/HDFS-12090
Commit: 7a68ac607c52c8a28dcd75a367ae77331787a3b4
Parents: 790c563
Author: Yiqun Lin <yqlin@apache.org>
Authored: Mon Jul 9 14:27:34 2018 +0800
Committer: Yiqun Lin <yqlin@apache.org>
Committed: Mon Jul 9 14:27:34 2018 +0800

----------------------------------------------------------------------
 .../hadoop-common/src/site/markdown/Metrics.md  | 39 ++++++++++++--------
 1 file changed, 24 insertions(+), 15 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/7a68ac60/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md b/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
index 676ab0b..2c7bd4d 100644
--- a/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
+++ b/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
@@ -159,14 +159,17 @@ Each metrics record contains tags such as ProcessName, SessionId, and
Hostname a
 | `TransactionsAvgTime` | Average time of Journal transactions in milliseconds |
 | `SyncsNumOps` | Total number of Journal syncs |
 | `SyncsAvgTime` | Average time of Journal syncs in milliseconds |
+| `SyncsTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of Journal sync time in milliseconds (*num* seconds granularity). Percentile measurement is
off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `TransactionsBatchedInSync` | Total number of Journal transactions batched in sync |
+| `TransactionsBatchedInSync`*num*`s(50/75/90/95/99)thPercentileCount` | The 50/75/90/95/99th
percentile of number of batched Journal transactions (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `StorageBlockReportNumOps` | Total number of processing block reports from individual storages
in DataNode |
 | `StorageBlockReportAvgTime` | Average time of processing block reports in milliseconds
|
+| `StorageBlockReport`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of block report processing time in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `CacheReportNumOps` | Total number of processing cache reports from DataNode |
 | `CacheReportAvgTime` | Average time of processing cache reports in milliseconds |
+| `CacheReport`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of cached report processing time in milliseconds (*num* seconds granularity). Percentile measurement
is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `SafeModeTime` | The interval between FSNameSystem starts and the last time safemode leaves
in milliseconds.  (sometimes not equal to the time in SafeMode, see [HDFS-5156](https://issues.apache.org/jira/browse/HDFS-5156))
|
 | `FsImageLoadTime` | Time loading FS Image at startup in milliseconds |
-| `FsImageLoadTime` | Time loading FS Image at startup in milliseconds |
 | `GetEditNumOps` | Total number of edits downloads from SecondaryNameNode |
 | `GetEditAvgTime` | Average edits download time in milliseconds |
 | `GetImageNumOps` | Total number of fsimage downloads from SecondaryNameNode |
@@ -177,22 +180,23 @@ Each metrics record contains tags such as ProcessName, SessionId, and
Hostname a
 | `NNStartedTimeInMillis`| NameNode start time in milliseconds |
 | `GenerateEDEKTimeNumOps` | Total number of generating EDEK |
 | `GenerateEDEKTimeAvgTime` | Average time of generating EDEK in milliseconds |
+| `GenerateEDEKTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of time spent in generating EDEK in milliseconds (*num* seconds granularity). Percentile measurement
is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `WarmUpEDEKTimeNumOps` | Total number of warming up EDEK |
 | `WarmUpEDEKTimeAvgTime` | Average time of warming up EDEK in milliseconds |
-| `ResourceCheckTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of NameNode resource check latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
-| `StorageBlockReport`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of storage block report latency in milliseconds. Percentile measurement is off
by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `WarmUpEDEKTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of time spent in warming up EDEK in milliseconds (*num* seconds granularity). Percentile measurement
is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `ResourceCheckTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of of NameNode resource check latency in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `EditLogTailTimeNumOps` | Total number of times the standby NameNode tailed the edit log
|
 | `EditLogTailTimeAvgTime` | Average time (in milliseconds) spent by standby NameNode in
tailing edit log |
-| `EditLogTailTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of time spent in tailing edit logs by standby NameNode, in milliseconds. Percentile measurement
is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `EditLogTailTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of time spent in tailing edit logs by standby NameNode in milliseconds (*num* seconds granularity).
Percentile measurement is off by default, by watching no intervals. The intervals are specified
by `dfs.metrics.percentiles.intervals`. |
 | `EditLogFetchTimeNumOps` | Total number of times the standby NameNode fetched remote edit
streams from journal nodes |
 | `EditLogFetchTimeAvgTime` | Average time (in milliseconds) spent by standby NameNode in
fetching remote edit streams from journal nodes |
-| `EditLogFetchTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of time spent in fetching edit streams from journal nodes by standby NameNode, in milliseconds.
Percentile measurement is off by default, by watching no intervals. The intervals are specified
by `dfs.metrics.percentiles.intervals`. |
+| `EditLogFetchTime`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of time spent in fetching edit streams from journal nodes by standby NameNode in milliseconds
(*num* seconds granularity). Percentile measurement is off by default, by watching no intervals.
The intervals are specified by `dfs.metrics.percentiles.intervals`. |
 | `NumEditLogLoadedNumOps` | Total number of times edits were loaded by standby NameNode
|
 | `NumEditLogLoadedAvgCount` | Average number of edits loaded by standby NameNode in each
edit log tailing |
-| `NumEditLogLoaded`*num*`s(50/75/90/95/99)thPercentileCount` | The 50/75/90/95/99th percentile
of number of edits loaded by standby NameNode in each edit log tailing. Percentile measurement
is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `NumEditLogLoaded`*num*`s(50/75/90/95/99)thPercentileCount` | The 50/75/90/95/99th percentile
of number of edits loaded by standby NameNode in each edit log tailing (*num* seconds granularity).
Percentile measurement is off by default, by watching no intervals. The intervals are specified
by `dfs.metrics.percentiles.intervals`. |
 | `EditLogTailIntervalNumOps` | Total number of intervals between edit log tailings by standby
NameNode |
 | `EditLogTailIntervalAvgTime` | Average time of intervals between edit log tailings by standby
NameNode in milliseconds |
-| `EditLogTailInterval`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of time between edit log tailings by standby NameNode, in milliseconds. Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `EditLogTailInterval`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of time between edit log tailings by standby NameNode in milliseconds (*num* seconds
granularity). Percentile measurement is off by default, by watching no intervals. The intervals
are specified by `dfs.metrics.percentiles.intervals`. |
 
 FSNamesystem
 ------------
@@ -338,13 +342,13 @@ Each metrics record contains tags such as SessionId and Hostname as
additional i
 | `RamDiskBlocksEvictedWithoutRead` | Total number of blocks evicted in memory without ever
being read from memory |
 | `RamDiskBlocksEvictionWindowMsNumOps` | Number of blocks evicted in memory|
 | `RamDiskBlocksEvictionWindowMsAvgTime` | Average time of blocks in memory before being
evicted in milliseconds |
-| `RamDiskBlocksEvictionWindows`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of latency between memory write and eviction in milliseconds. Percentile measurement
is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `RamDiskBlocksEvictionWindows`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of latency between memory write and eviction in milliseconds (*num* seconds granularity).
Percentile measurement is off by default, by watching no intervals. The intervals are specified
by `dfs.metrics.percentiles.intervals`. |
 | `RamDiskBlocksLazyPersisted` | Total number of blocks written to disk by lazy writer |
 | `RamDiskBlocksDeletedBeforeLazyPersisted` | Total number of blocks deleted by application
before being persisted to disk |
 | `RamDiskBytesLazyPersisted` | Total number of bytes written to disk by lazy writer |
 | `RamDiskBlocksLazyPersistWindowMsNumOps` | Number of blocks written to disk by lazy writer
|
 | `RamDiskBlocksLazyPersistWindowMsAvgTime` | Average time of blocks written to disk by lazy
writer in milliseconds |
-| `RamDiskBlocksLazyPersistWindows`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of latency between memory write and disk persist in milliseconds. Percentile measurement
is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `RamDiskBlocksLazyPersistWindows`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of latency between memory write and disk persist in milliseconds (*num* seconds
granularity). Percentile measurement is off by default, by watching no intervals. The intervals
are specified by `dfs.metrics.percentiles.intervals`. |
 | `FsyncCount` | Total number of fsync |
 | `VolumeFailures` | Total number of volume failures occurred |
 | `ReadBlockOpNumOps` | Total number of read operations |
@@ -371,14 +375,19 @@ Each metrics record contains tags such as SessionId and Hostname as
additional i
 | `CacheReportsAvgTime` | Average time of cache report operations in milliseconds |
 | `PacketAckRoundTripTimeNanosNumOps` | Total number of ack round trip |
 | `PacketAckRoundTripTimeNanosAvgTime` | Average time from ack send to receive minus the
downstream ack time in nanoseconds |
+| `PacketAckRoundTripTimeNanos`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile latency from ack send to receive minus the downstream ack time in nanoseconds (*num*
seconds granularity). Percentile measurement is off by default, by watching no intervals.
The intervals are specified by `dfs.metrics.percentiles.intervals`. |
 | `FlushNanosNumOps` | Total number of flushes |
 | `FlushNanosAvgTime` | Average flush time in nanoseconds |
+| `FlushNanos`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
flush time in nanoseconds (*num* seconds granularity). Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `FsyncNanosNumOps` | Total number of fsync |
 | `FsyncNanosAvgTime` | Average fsync time in nanoseconds |
+| `FsyncNanos`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
fsync time in nanoseconds (*num* seconds granularity). Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `SendDataPacketBlockedOnNetworkNanosNumOps` | Total number of sending packets |
 | `SendDataPacketBlockedOnNetworkNanosAvgTime` | Average waiting time of sending packets
in nanoseconds |
+| `SendDataPacketBlockedOnNetworkNanos`*num*`s(50/75/90/95/99)thPercentileLatency` | The
50/75/90/95/99th percentile waiting time of sending packets in nanoseconds (*num* seconds
granularity). Percentile measurement is off by default, by watching no intervals. The intervals
are specified by `dfs.metrics.percentiles.intervals`. |
 | `SendDataPacketTransferNanosNumOps` | Total number of sending packets |
 | `SendDataPacketTransferNanosAvgTime` | Average transfer time of sending packets in nanoseconds
|
+| `SendDataPacketTransferNanos`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile transfer time of sending packets in nanoseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `TotalWriteTime`| Total number of milliseconds spent on write operation |
 | `TotalReadTime` | Total number of milliseconds spent on read operation |
 | `RemoteBytesRead` | Number of bytes read by remote clients |
@@ -410,23 +419,23 @@ contains tags such as Hostname as additional information along with
metrics.
 | `TotalMetadataOperations` | Total number (monotonically increasing) of metadata operations.
Metadata operations include stat, list, mkdir, delete, move, open and posix_fadvise. |
 | `MetadataOperationRateNumOps` | The number of metadata operations within an interval time
of metric |
 | `MetadataOperationRateAvgTime` | Mean time of metadata operations in milliseconds |
-| `MetadataOperationLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of metadata operations latency in milliseconds. Percentile measurement is off by
default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `MetadataOperationLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th
percentile of metadata operations latency in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `TotalDataFileIos` | Total number (monotonically increasing) of data file io operations
|
 | `DataFileIoRateNumOps` | The number of data file io operations within an interval time
of metric |
 | `DataFileIoRateAvgTime` | Mean time of data file io operations in milliseconds |
-| `DataFileIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of data file io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `DataFileIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of data file io operations latency in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `FlushIoRateNumOps` | The number of file flush io operations within an interval time of
metric |
 | `FlushIoRateAvgTime` | Mean time of file flush io operations in milliseconds |
-| `FlushIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file flush io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `FlushIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file flush io operations latency in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `SyncIoRateNumOps` | The number of file sync io operations within an interval time of metric
|
 | `SyncIoRateAvgTime` | Mean time of file sync io operations in milliseconds |
-| `SyncIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file sync io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `SyncIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file sync io operations latency in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `ReadIoRateNumOps` | The number of file read io operations within an interval time of metric
|
 | `ReadIoRateAvgTime` | Mean time of file read io operations in milliseconds |
-| `ReadIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file read io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `ReadIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file read io operations latency in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `WriteIoRateNumOps` | The number of file write io operations within an interval time of
metric |
 | `WriteIoRateAvgTime` | Mean time of file write io operations in milliseconds |
-| `WriteIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file write io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `WriteIoLatency`*num*`s(50/75/90/95/99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file write io operations latency in milliseconds (*num* seconds granularity). Percentile
measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
 | `TotalFileIoErrors` | Total number (monotonically increasing) of file io error operations
|
 | `FileIoErrorRateNumOps` | The number of file io error operations within an interval time
of metric |
 | `FileIoErrorRateAvgTime` | It measures the mean time in milliseconds from the start of
an operation to hitting a failure |


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message