hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From xg...@apache.org
Subject [07/21] hadoop git commit: HDFS-11282. Document the missing metrics of DataNode Volume IO operations. Contributed by Yiqun Lin.
Date Mon, 09 Jan 2017 21:30:02 GMT
HDFS-11282. Document the missing metrics of DataNode Volume IO operations. Contributed by Yiqun
Lin.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/4a659ff4
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/4a659ff4
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/4a659ff4

Branch: refs/heads/YARN-5734
Commit: 4a659ff40fca7c263d62ac7514afc100a4dbb1ed
Parents: 0b8a7c1
Author: Arpit Agarwal <arp@apache.org>
Authored: Thu Jan 5 19:34:32 2017 -0800
Committer: Arpit Agarwal <arp@apache.org>
Committed: Thu Jan 5 19:34:32 2017 -0800

----------------------------------------------------------------------
 .../hadoop-common/src/site/markdown/Metrics.md  | 31 ++++++++++++++++++++
 1 file changed, 31 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/4a659ff4/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
----------------------------------------------------------------------
diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md b/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
index 1d5ea5e..541f5dc 100644
--- a/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
+++ b/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
@@ -323,6 +323,37 @@ Each metrics record contains tags such as SessionId and Hostname as additional
i
 | `RemoteBytesWritten` | Number of bytes written by remote clients |
 | `BPServiceActorInfo` | The information about a block pool service actor |
 
+FsVolume
+--------
+
+Per-volume metrics contain Datanode Volume IO related statistics. Per-volume metrics are
off by default. They can be enbabled by setting `dfs.datanode.fileio.events.class` to **org.apache.hadoop.hdfs.server.datanode.ProfilingFileIoEvents**,
but enabling per-volume metrics may have a performance impact. Each metrics record contains
tags such as Hostname as additional information along with metrics.
+
+| Name | Description |
+|:---- |:---- |
+| `TotalMetadataOperations` | Total number (monotonically increasing) of metadata operations.
Metadata operations include stat, list, mkdir, delete, move, open and posix_fadvise. |
+| `MetadataOperationRateNumOps` | The number of metadata operations within an interval time
of metric |
+| `MetadataOperationRateAvgTime` | Mean time of metadata operations in milliseconds |
+| `MetadataOperationLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th
percentile of metadata operations latency in milliseconds. Percentile measurement is off by
default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `TotalDataFileIos` | Total number (monotonically increasing) of data file io operations
|
+| `DataFileIoRateNumOps` | The number of data file io operations within an interval time
of metric |
+| `DataFileIoRateAvgTime` | Mean time of data file io operations in milliseconds |
+| `DataFileIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile
of data file io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `FlushIoRateNumOps` | The number of file flush io operations within an interval time of
metric |
+| `FlushIoRateAvgTime` | Mean time of file flush io operations in milliseconds |
+| `FlushIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file flush io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `SyncIoRateNumOps` | The number of file sync io operations within an interval time of metric
|
+| `SyncIoRateAvgTime` | Mean time of file sync io operations in milliseconds |
+| `SyncIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file sync io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `ReadIoRateNumOps` | The number of file read io operations within an interval time of metric
|
+| `ReadIoRateAvgTime` | Mean time of file read io operations in milliseconds |
+| `ReadIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file read io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `WriteIoRateNumOps` | The number of file write io operations within an interval time of
metric |
+| `WriteIoRateAvgTime` | Mean time of file write io operations in milliseconds |
+| `WriteIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile
of file write io operations latency in milliseconds. Percentile measurement is off by default,
by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`.
|
+| `TotalFileIoErrors` | Total number (monotonically increasing) of file io error operations
|
+| `FileIoErrorRateNumOps` | The number of file io error operations within an interval time
of metric |
+| `FileIoErrorRateAvgTime` | It measures the mean time in milliseconds from the start of
an operation to hitting a failure |
+
 yarn context
 ============
 


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message