hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ka...@apache.org
Subject hadoop git commit: YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha)
Date Fri, 08 May 2015 23:31:18 GMT
Repository: hadoop
Updated Branches:
  refs/heads/trunk 59995cec4 -> 96473bdc2


YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via
kasha)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/96473bdc
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/96473bdc
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/96473bdc

Branch: refs/heads/trunk
Commit: 96473bdc2b090c13708fd467fd626621ef1d47eb
Parents: 59995ce
Author: Karthik Kambatla <kasha@apache.org>
Authored: Fri May 8 16:31:05 2015 -0700
Committer: Karthik Kambatla <kasha@apache.org>
Committed: Fri May 8 16:31:14 2015 -0700

----------------------------------------------------------------------
 hadoop-yarn-project/CHANGES.txt                 |   2 +
 .../src/site/markdown/ResourceManagerRest.md    | 306 ++++++++++++++++++-
 2 files changed, 307 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/96473bdc/hadoop-yarn-project/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt
index 02226db..9b75474 100644
--- a/hadoop-yarn-project/CHANGES.txt
+++ b/hadoop-yarn-project/CHANGES.txt
@@ -209,6 +209,8 @@ Release 2.8.0 - UNRELEASED
     YARN-2331. Distinguish shutdown during supervision vs. shutdown for
     rolling upgrade. (Jason Lowe via xgong)
 
+    YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik
via kasha)
+
   OPTIMIZATIONS
 
     YARN-3339. TestDockerContainerExecutor should pull a single image and not

http://git-wip-us.apache.org/repos/asf/hadoop/blob/96473bdc/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
index b1591bb..d5df8ba 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
@@ -278,7 +278,7 @@ Response Body:
 Cluster Scheduler API
 ---------------------
 
-A scheduler resource contains information about the current scheduler configured in a cluster.
It currently supports both the Fifo and Capacity Scheduler. You will get different information
depending on which scheduler is configured so be sure to look at the type information.
+A scheduler resource contains information about the current scheduler configured in a cluster.
It currently supports the Fifo, Capacity and Fair Scheduler. You will get different information
depending on which scheduler is configured so be sure to look at the type information.
 
 ### URI
 
@@ -984,6 +984,310 @@ Response Body:
 </scheduler>
 ```
 
+### Fair Scheduler API
+
+### Elements of the *schedulerInfo* object
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| type | string | Scheduler type - fairScheduler |
+| rootQueue | The root queue object | A collection of root queue resources |
+
+### Elements of the root queue object
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| maxApps | int | The maximum number of applications the queue can have |
+| minResources | A single resource object | The configured minimum resources that are guaranteed
to the queue |
+| maxResources | A single resource object | The configured maximum resources that are allowed
to the queue |
+| usedResources | A single resource object | The sum of resources allocated to containers
within the queue |
+| fairResources | A single resource object | The queue's fair share of resources |
+| clusterResources | A single resource object | The capacity of the cluster |
+| queueName | string | The name of the queue |
+| schedulingPolicy | string | The name of the scheduling policy used by the queue |
+| childQueues | array of queues(JSON)/queue objects(XML) | A collection of sub-queue information
|
+
+### Elements of the queues object for a Leaf queue - contains all elements in parent plus
the following
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| type | string | type of the queue - fairSchedulerLeafQueueInfo |
+| numActiveApps | int | The number of active applications in this queue |
+| numPendingApps | int | The number of pending applications in this queue |
+
+### Elements of the resource object for resourcesUsed in queues
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| memory | int | The amount of memory used (in MB) |
+| vCores | int | The number of virtual cores |
+
+#### Response Examples
+
+**JSON response**
+
+HTTP Request:
+
+      GET http://<rm http address:port>/ws/v1/cluster/scheduler
+
+Response Header:
+
+      HTTP/1.1 200 OK
+      Content-Type: application/json
+      Transfer-Encoding: chunked
+      Server: Jetty(6.1.26)
+
+Response Body:
+
+```json
+{
+    "scheduler": {
+        "schedulerInfo": {
+            "rootQueue": {
+                "childQueues": [
+                    {
+                        "clusterResources": {
+                            "memory": 8192,
+                            "vCores": 8
+                        },
+                        "fairResources": {
+                            "memory": 0,
+                            "vCores": 0
+                        },
+                        "maxApps": 2147483647,
+                        "maxResources": {
+                            "memory": 8192,
+                            "vCores": 8
+                        },
+                        "minResources": {
+                            "memory": 0,
+                            "vCores": 0
+                        },
+                        "numActiveApps": 0,
+                        "numPendingApps": 0,
+                        "queueName": "root.default",
+                        "schedulingPolicy": "fair",
+                        "type": "fairSchedulerLeafQueueInfo",
+                        "usedResources": {
+                            "memory": 0,
+                            "vCores": 0
+                        }
+                    },
+                    {
+                        "childQueues": {
+                            "clusterResources": {
+                                "memory": 8192,
+                                "vCores": 8
+                            },
+                            "fairResources": {
+                                "memory": 10000,
+                                "vCores": 0
+                            },
+                            "maxApps": 2147483647,
+                            "maxResources": {
+                                "memory": 8192,
+                                "vCores": 8
+                            },
+                            "minResources": {
+                                "memory": 5000,
+                                "vCores": 0
+                            },
+                            "numActiveApps": 0,
+                            "numPendingApps": 0,
+                            "queueName": "root.sample_queue.sample_sub_queue",
+                            "schedulingPolicy": "fair",
+                            "type": [
+                                "fairSchedulerLeafQueueInfo"
+                            ],
+                            "usedResources": {
+                                "memory": 0,
+                                "vCores": 0
+                            }
+                        },
+                        "clusterResources": {
+                            "memory": 8192,
+                            "vCores": 8
+                        },
+                        "fairResources": {
+                            "memory": 10000,
+                            "vCores": 0
+                        },
+                        "maxApps": 50,
+                        "maxResources": {
+                            "memory": 8192,
+                            "vCores": 0
+                        },
+                        "minResources": {
+                            "memory": 10000,
+                            "vCores": 0
+                        },
+                        "queueName": "root.sample_queue",
+                        "schedulingPolicy": "fair",
+                        "usedResources": {
+                            "memory": 0,
+                            "vCores": 0
+                        }
+                    }
+                ],
+                "clusterResources": {
+                    "memory": 8192,
+                    "vCores": 8
+                },
+                "fairResources": {
+                    "memory": 8192,
+                    "vCores": 8
+                },
+                "maxApps": 2147483647,
+                "maxResources": {
+                    "memory": 8192,
+                    "vCores": 8
+                },
+                "minResources": {
+                    "memory": 0,
+                    "vCores": 0
+                },
+                "queueName": "root",
+                "schedulingPolicy": "fair",
+                "usedResources": {
+                    "memory": 0,
+                    "vCores": 0
+                }
+            },
+            "type": "fairScheduler"
+        }
+    }
+}
+```
+
+**XML response**
+
+HTTP Request:
+
+      GET http://<rm http address:port>/ws/v1/cluster/scheduler
+      Accept: application/xml
+
+Response Header:
+
+      HTTP/1.1 200 OK
+      Content-Type: application/xml
+      Content-Length: 2321 
+      Server: Jetty(6.1.26)
+
+Response Body:
+
+```xml
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<scheduler>
+  <schedulerInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="fairScheduler">
+    <rootQueue>
+      <maxApps>2147483647</maxApps>
+      <minResources>
+        <memory>0</memory>
+        <vCores>0</vCores>
+      </minResources>
+      <maxResources>
+        <memory>8192</memory>
+        <vCores>8</vCores>
+      </maxResources>
+      <usedResources>
+        <memory>0</memory>
+        <vCores>0</vCores>
+      </usedResources>
+      <fairResources>
+        <memory>8192</memory>
+        <vCores>8</vCores>
+      </fairResources>
+      <clusterResources>
+        <memory>8192</memory>
+        <vCores>8</vCores>
+      </clusterResources>
+      <queueName>root</queueName>
+      <schedulingPolicy>fair</schedulingPolicy>
+      <childQueues xsi:type="fairSchedulerLeafQueueInfo">
+        <maxApps>2147483647</maxApps>
+        <minResources>
+          <memory>0</memory>
+          <vCores>0</vCores>
+        </minResources>
+        <maxResources>
+          <memory>8192</memory>
+          <vCores>8</vCores>
+        </maxResources>
+        <usedResources>
+          <memory>0</memory>
+          <vCores>0</vCores>
+        </usedResources>
+        <fairResources>
+          <memory>0</memory>
+          <vCores>0</vCores>
+        </fairResources>
+        <clusterResources>
+          <memory>8192</memory>
+          <vCores>8</vCores>
+        </clusterResources>
+        <queueName>root.default</queueName>
+        <schedulingPolicy>fair</schedulingPolicy>
+        <numPendingApps>0</numPendingApps>
+        <numActiveApps>0</numActiveApps>
+      </childQueues>
+      <childQueues>
+        <maxApps>50</maxApps>
+        <minResources>
+          <memory>10000</memory>
+          <vCores>0</vCores>
+        </minResources>
+        <maxResources>
+          <memory>8192</memory>
+          <vCores>0</vCores>
+        </maxResources>
+        <usedResources>
+          <memory>0</memory>
+          <vCores>0</vCores>
+        </usedResources>
+        <fairResources>
+          <memory>10000</memory>
+          <vCores>0</vCores>
+        </fairResources>
+        <clusterResources>
+          <memory>8192</memory>
+          <vCores>8</vCores>
+        </clusterResources>
+        <queueName>root.sample_queue</queueName>
+        <schedulingPolicy>fair</schedulingPolicy>
+        <childQueues xsi:type="fairSchedulerLeafQueueInfo">
+          <maxApps>2147483647</maxApps>
+          <minResources>
+            <memory>5000</memory>
+            <vCores>0</vCores>
+          </minResources>
+          <maxResources>
+            <memory>8192</memory>
+            <vCores>8</vCores>
+          </maxResources>
+          <usedResources>
+            <memory>0</memory>
+            <vCores>0</vCores>
+          </usedResources>
+          <fairResources>
+            <memory>10000</memory>
+            <vCores>0</vCores>
+          </fairResources>
+          <clusterResources>
+            <memory>8192</memory>
+            <vCores>8</vCores>
+          </clusterResources>
+          <queueName>root.sample_queue.sample_sub_queue</queueName>
+          <schedulingPolicy>fair</schedulingPolicy>
+          <numPendingApps>0</numPendingApps>
+          <numActiveApps>0</numActiveApps>
+        </childQueues>
+      </childQueues>
+    </rootQueue>
+  </schedulerInfo>
+</scheduler>
+```
+
+
 Cluster Applications API
 ------------------------
 


Mime
View raw message