hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From yq...@apache.org
Subject hadoop git commit: HDFS-11963. Ozone: Documentation: Add getting started page (addendum). Contributed by Yiqun Lin.
Date Wed, 21 Jun 2017 02:33:20 GMT
Repository: hadoop
Updated Branches:
  refs/heads/HDFS-7240 66552374e -> b9a6441e1


HDFS-11963. Ozone: Documentation: Add getting started page (addendum). Contributed by Yiqun
Lin.


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/b9a6441e
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/b9a6441e
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/b9a6441e

Branch: refs/heads/HDFS-7240
Commit: b9a6441e1b297dd5f6ee8eaebf7ec3f7ec34c019
Parents: 6655237
Author: Yiqun Lin <yqlin@apache.org>
Authored: Wed Jun 21 10:32:35 2017 +0800
Committer: Yiqun Lin <yqlin@apache.org>
Committed: Wed Jun 21 10:32:35 2017 +0800

----------------------------------------------------------------------
 .../src/site/markdown/OzoneMetrics.md           | 128 +++++++++++++++++++
 .../src/site/markdown/Ozonemetrics.md           | 128 -------------------
 hadoop-project/src/site/site.xml                |   3 +-
 3 files changed, 129 insertions(+), 130 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/b9a6441e/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/OzoneMetrics.md
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/OzoneMetrics.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/OzoneMetrics.md
new file mode 100644
index 0000000..21e3474
--- /dev/null
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/OzoneMetrics.md
@@ -0,0 +1,128 @@
+<!---
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License. See accompanying LICENSE file.
+-->
+
+
+
+HDFS Ozone Metrics
+===============
+
+<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
+
+Overview
+--------
+
+The container metrics that is used in HDFS Ozone.
+
+### Storage Container Metrics
+
+The metrics for various storage container operations in HDFS Ozone.
+
+Storage container is an optional service that can be enabled by setting
+'ozone.enabled' to true.
+These metrics are only available when ozone is enabled.
+
+Storage Container Metrics maintains a set of generic metrics for all
+container RPC calls that can be made to a datandoe/container.
+
+Along with the total number of RPC calls containers maintain a set of metrics
+for each RPC call. Following is the set of counters maintained for each RPC
+operation.
+
+*Total number of operation* - We maintain an array which counts how
+many times a specific operation has been performed.
+Eg.`NumCreateContainer` tells us how many times create container has been
+invoked on this datanode.
+
+*Number of bytes involved in a specific command* - This is an array that is
+maintained for all operations, but makes sense only for read and write
+operations.
+
+While it is possible to read the bytes in update container, it really makes
+no sense, since no data stream involved. Users are advised to use this
+metric only when it makes sense. Eg. `BytesReadChunk` -- Tells us how
+many bytes have been read from this data using Read Chunk operation.
+
+*Average Latency of each operation* - The average latency of the operation.
+Eg. `LatencyCreateContainerAvgTime` - This tells us the average latency of
+Create Container.
+
+*Quantiles for each of these operations* - The 50/75/90/95/99th percentile
+of these operations. Eg. `CreateContainerNanos60s50thPercentileLatency` --
+gives latency of the create container operations at the 50th percentile latency
+(1 minute granularity). We report 50th, 75th, 90th, 95th and 99th percentile
+for all RPCs.
+
+So this leads to the containers reporting these counters for each of these
+RPC operations.
+
+| Name | Description |
+|:---- |:---- |
+| `NumOps` | Total number of container operations |
+| `CreateContainer` | Create container operation |
+| `ReadContainer` | Read container operation |
+| `UpdateContainer` | Update container operations |
+| `DeleteContainer` | Delete container operations |
+| `ListContainer` | List container operations |
+| `PutKey` | Put key operations |
+| `GetKey` | Get key operations |
+| `DeleteKey` | Delete key operations |
+| `ListKey` | List key operations |
+| `ReadChunk` | Read chunk operations |
+| `DeleteChunk` | Delete chunk operations |
+| `WriteChunk` | Write chunk operations|
+| `ListChunk` | List chunk operations |
+| `CompactChunk` | Compact chunk operations |
+| `PutSmallFile` | Put small file operations |
+| `GetSmallFile` | Get small file operations |
+| `CloseContainer` | Close container operations |
+
+### Key Space Metrics
+
+The metrics for various key space manager operations in HDFS Ozone.
+
+key space manager (KSM) is a service that similar to the Namenode in HDFS.
+In the current design of KSM, it maintains metadata of all volumes, buckets and keys.
+These metrics are only available when ozone is enabled.
+
+Following is the set of counters maintained for each key space operation.
+
+*Total number of operation* - We maintain an array which counts how
+many times a specific operation has been performed.
+Eg.`NumVolumeCreate` tells us how many times create volume has been
+invoked in KSM.
+
+*Total number of failed operation* - This type operation is opposite to the above
+operation.
+Eg.`NumVolumeCreateFails` tells us how many times create volume has been invoked
+failed in KSM.
+
+Following are the counters for each of key space operations.
+
+| Name | Description |
+|:---- |:---- |
+| `VolumeCreate` | Create volume operation |
+| `VolumeUpdates` | Update volume property operation |
+| `VolumeInfos` | Get volume information operation |
+| `VolumeCheckAccesses` | Check volume access operation |
+| `VolumeDeletes` | Delete volume operation |
+| `VolumeLists` | List volume operation |
+| `BucketCreates` | Create bucket operation |
+| `BucketInfos` | Get bucket information operation |
+| `BucketUpdates` | Update bucket property operation |
+| `BucketDeletes` | Delete bucket operation |
+| `BucketLists` | List bucket operation |
+| `KeyAllocate` | Allocate key operation |
+| `KeyLookup` | Look up key operation |
+| `KeyDeletes` | Delete key operation |
+| `KeyLists` | List key operation |
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/hadoop/blob/b9a6441e/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Ozonemetrics.md
----------------------------------------------------------------------
diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Ozonemetrics.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Ozonemetrics.md
deleted file mode 100644
index 21e3474..0000000
--- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Ozonemetrics.md
+++ /dev/null
@@ -1,128 +0,0 @@
-<!---
-  Licensed under the Apache License, Version 2.0 (the "License");
-  you may not use this file except in compliance with the License.
-  You may obtain a copy of the License at
-
-   http://www.apache.org/licenses/LICENSE-2.0
-
-  Unless required by applicable law or agreed to in writing, software
-  distributed under the License is distributed on an "AS IS" BASIS,
-  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-  See the License for the specific language governing permissions and
-  limitations under the License. See accompanying LICENSE file.
--->
-
-
-
-HDFS Ozone Metrics
-===============
-
-<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
-
-Overview
---------
-
-The container metrics that is used in HDFS Ozone.
-
-### Storage Container Metrics
-
-The metrics for various storage container operations in HDFS Ozone.
-
-Storage container is an optional service that can be enabled by setting
-'ozone.enabled' to true.
-These metrics are only available when ozone is enabled.
-
-Storage Container Metrics maintains a set of generic metrics for all
-container RPC calls that can be made to a datandoe/container.
-
-Along with the total number of RPC calls containers maintain a set of metrics
-for each RPC call. Following is the set of counters maintained for each RPC
-operation.
-
-*Total number of operation* - We maintain an array which counts how
-many times a specific operation has been performed.
-Eg.`NumCreateContainer` tells us how many times create container has been
-invoked on this datanode.
-
-*Number of bytes involved in a specific command* - This is an array that is
-maintained for all operations, but makes sense only for read and write
-operations.
-
-While it is possible to read the bytes in update container, it really makes
-no sense, since no data stream involved. Users are advised to use this
-metric only when it makes sense. Eg. `BytesReadChunk` -- Tells us how
-many bytes have been read from this data using Read Chunk operation.
-
-*Average Latency of each operation* - The average latency of the operation.
-Eg. `LatencyCreateContainerAvgTime` - This tells us the average latency of
-Create Container.
-
-*Quantiles for each of these operations* - The 50/75/90/95/99th percentile
-of these operations. Eg. `CreateContainerNanos60s50thPercentileLatency` --
-gives latency of the create container operations at the 50th percentile latency
-(1 minute granularity). We report 50th, 75th, 90th, 95th and 99th percentile
-for all RPCs.
-
-So this leads to the containers reporting these counters for each of these
-RPC operations.
-
-| Name | Description |
-|:---- |:---- |
-| `NumOps` | Total number of container operations |
-| `CreateContainer` | Create container operation |
-| `ReadContainer` | Read container operation |
-| `UpdateContainer` | Update container operations |
-| `DeleteContainer` | Delete container operations |
-| `ListContainer` | List container operations |
-| `PutKey` | Put key operations |
-| `GetKey` | Get key operations |
-| `DeleteKey` | Delete key operations |
-| `ListKey` | List key operations |
-| `ReadChunk` | Read chunk operations |
-| `DeleteChunk` | Delete chunk operations |
-| `WriteChunk` | Write chunk operations|
-| `ListChunk` | List chunk operations |
-| `CompactChunk` | Compact chunk operations |
-| `PutSmallFile` | Put small file operations |
-| `GetSmallFile` | Get small file operations |
-| `CloseContainer` | Close container operations |
-
-### Key Space Metrics
-
-The metrics for various key space manager operations in HDFS Ozone.
-
-key space manager (KSM) is a service that similar to the Namenode in HDFS.
-In the current design of KSM, it maintains metadata of all volumes, buckets and keys.
-These metrics are only available when ozone is enabled.
-
-Following is the set of counters maintained for each key space operation.
-
-*Total number of operation* - We maintain an array which counts how
-many times a specific operation has been performed.
-Eg.`NumVolumeCreate` tells us how many times create volume has been
-invoked in KSM.
-
-*Total number of failed operation* - This type operation is opposite to the above
-operation.
-Eg.`NumVolumeCreateFails` tells us how many times create volume has been invoked
-failed in KSM.
-
-Following are the counters for each of key space operations.
-
-| Name | Description |
-|:---- |:---- |
-| `VolumeCreate` | Create volume operation |
-| `VolumeUpdates` | Update volume property operation |
-| `VolumeInfos` | Get volume information operation |
-| `VolumeCheckAccesses` | Check volume access operation |
-| `VolumeDeletes` | Delete volume operation |
-| `VolumeLists` | List volume operation |
-| `BucketCreates` | Create bucket operation |
-| `BucketInfos` | Get bucket information operation |
-| `BucketUpdates` | Update bucket property operation |
-| `BucketDeletes` | Delete bucket operation |
-| `BucketLists` | List bucket operation |
-| `KeyAllocate` | Allocate key operation |
-| `KeyLookup` | Look up key operation |
-| `KeyDeletes` | Delete key operation |
-| `KeyLists` | List key operation |
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/hadoop/blob/b9a6441e/hadoop-project/src/site/site.xml
----------------------------------------------------------------------
diff --git a/hadoop-project/src/site/site.xml b/hadoop-project/src/site/site.xml
index 9fb8f30..889f62e 100644
--- a/hadoop-project/src/site/site.xml
+++ b/hadoop-project/src/site/site.xml
@@ -106,7 +106,7 @@
     <menu name="Ozone" inherit="top">
       <item name="Getting Started" href="hadoop-project-dist/hadoop-hdfs/OzoneGettingStarted.html"/>
       <item name="Commands Reference" href="hadoop-project-dist/hadoop-hdfs/OzoneCommandShell.html"/>
-      <item name="Ozone Metrics" href="hadoop-project-dist/hadoop-hdfs/Ozonemetrics.html"/>
+      <item name="Ozone Metrics" href="hadoop-project-dist/hadoop-hdfs/OzoneMetrics.html"/>
     </menu>
 
     <menu name="MapReduce" inherit="top">
@@ -186,7 +186,6 @@
       <item name="Java API docs" href="api/index.html"/>
       <item name="Unix Shell API" href="hadoop-project-dist/hadoop-common/UnixShellAPI.html"/>
       <item name="Metrics" href="hadoop-project-dist/hadoop-common/Metrics.html"/>
-      <item name="Ozone Metrics" href="hadoop-project-dist/hadoop-hdfs/Ozonemetrics.html"/>
     </menu>
     
     <menu name="Configuration" inherit="top">


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message