hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wan...@apache.org
Subject [2/7] hadoop git commit: YARN-8301. Added YARN service upgrade instructions. Contributed by Chandni Singh
Date Tue, 31 Jul 2018 19:26:27 GMT
YARN-8301.  Added YARN service upgrade instructions.
            Contributed by Chandni Singh

(cherry picked from commit 10014a4d88f239d3c072e51bc0739cba1fca9406)
(cherry picked from commit 4f2a129f2e2326ca28659d93b412cf8649ed5025)


Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/7cb37ed6
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/7cb37ed6
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/7cb37ed6

Branch: refs/heads/branch-3.1.1
Commit: 7cb37ed66bc830af41b8fb87207023e74ac5386f
Parents: 0c3d3bc
Author: Eric Yang <eyang@apache.org>
Authored: Fri Jul 20 19:46:35 2018 -0400
Committer: Wangda Tan <wangda@apache.org>
Committed: Tue Jul 31 12:06:23 2018 -0700

----------------------------------------------------------------------
 .../src/site/markdown/yarn-service/Overview.md  |   4 +-
 .../markdown/yarn-service/ServiceUpgrade.md     | 197 +++++++++++++++++++
 2 files changed, 198 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/hadoop/blob/7cb37ed6/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/Overview.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/Overview.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/Overview.md
index 8e2bf9a..041b0ee 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/Overview.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/Overview.md
@@ -56,6 +56,4 @@ The benefits of combining these workloads are two-fold:
 * [Registry DNS](RegistryDNS.html): Deep dives into the Registry DNS internals.
 * [Examples](Examples.html): List some example service definitions (`Yarnfile`).
 * [Configurations](Configurations.html): Describes how to configure the custom services on
YARN.
-
-
- 
+* [Service Upgrade](ServiceUpgrade.html): Describes how to upgrade a YARN service which is
an experimental feature.

http://git-wip-us.apache.org/repos/asf/hadoop/blob/7cb37ed6/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/ServiceUpgrade.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/ServiceUpgrade.md
b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/ServiceUpgrade.md
new file mode 100644
index 0000000..839be22
--- /dev/null
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/yarn-service/ServiceUpgrade.md
@@ -0,0 +1,197 @@
+<!---
+  Licensed under the Apache License, Version 2.0 (the "License");
+  you may not use this file except in compliance with the License.
+  You may obtain a copy of the License at
+
+   http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License. See accompanying LICENSE file.
+-->
+
+# Service Upgrade (Experimental Feature - Tech Preview)
+
+Yarn service provides a way of upgrading/downgrading long running applications without
+shutting down the application to minimize the downtime during this process. This is
+an experimental feature which is currently not enabled by default.
+
+## Overview
+
+Upgrading a Yarn Service is a 3 steps (or 2 steps when auto-finalization of
+upgrade is chosen) process:
+
+1. Initiate service upgrade.\
+This step involves providing the service spec of the newer version of the service.
+Once, the service upgrade is initiated, the state of the service is changed to
+`UPGRADING`.
+
+2. Upgrade component instances.\
+This step involves triggering upgrade of individual component instance.
+By providing an API to upgrade at instance level, users can orchestrate upgrade
+of the entire service in any order which is relevant for the service.\
+In addition, there are APIs to upgrade multiple instances, all instances of a
+component, and all instances of multiple components.
+
+3. Finalize upgrade.\
+This step involves finalization of upgrade. With an explicit step to finalize the
+upgrade, users have a chance to cancel current upgrade in progress. When the
+user chose to cancel, the service will make the best effort to revert to the
+previous version.\
+\
+When the upgrade is finalized, the old service definition is
+overwritten by the new service definition and the service state changes to `STABLE`.\
+A service can be auto-finalized when the upgrade is initialized with
+`-autoFinalize` option. With auto-finalization, when all the component-instances of
+the service have been upgraded, finalization will be performed automatically by the
+service framework.\
+\
+**NOTE**: Cancel of upgrade is not implemented yet.
+
+## Upgrade Example
+This example shows upgrade of sleeper service. Below is the sleeper service
+definition
+
+```
+{
+  "name": "sleeper-service",
+  "components" :
+    [
+      {
+        "name": "sleeper",
+        "version": "1.0.0",
+        "number_of_containers": 1,
+        "launch_command": "sleep 900000",
+        "resource": {
+          "cpus": 1,
+          "memory": "256"
+       }
+      }
+    ]
+}
+```
+Assuming, user launched an instance of sleeper service named as `my-sleeper`:
+```
+{
+  "components":
+    [
+      {
+        "configuration": {...},
+        "containers":
+          [
+            {
+              "bare_host": "0.0.0.0",
+              "component_instance_name": "sleeper-0",
+              "hostname": "example.local",
+              "id": "container_1531508836237_0002_01_000002",
+              "ip": "0.0.0.0",
+              "launch_time": 1531941023675,
+              "state": "READY"
+            },
+            {
+              "bare_host": "0.0.0.0",
+              "component_instance_name": "sleeper-1",
+              "hostname": "example.local",
+              "id": "container_1531508836237_0002_01_000003",
+              "ip": "0.0.0.0",
+              "launch_time": 1531941024680,
+              "state": "READY"
+            }
+          ],
+        "dependencies": [],
+        "launch_command": "sleep 900000",
+        "name": "sleeper",
+        "number_of_containers": 2,
+        "quicklinks": [],
+        "resource": {...},
+        "restart_policy": "ALWAYS",
+        "run_privileged_container": false,
+        "state": "STABLE"
+      }
+    ],
+  "configuration": {...},
+  "id": "application_1531508836237_0002",
+  "kerberos_principal": {},
+  "lifetime": -1,
+  "name": "my-sleeper",
+  "quicklinks": {},
+  "state": "STABLE",
+  "version": "1.0.0"
+}
+```
+
+### Enable Service Upgrade
+Below is the configuration in `yarn-site.xml` required for enabling service
+upgrade.
+
+```
+  <property>
+    <name>yarn.service.upgrade.enabled</name>
+    <value>true</value>
+  </property>
+```
+
+### Initiate Upgrade
+User can initiate upgrade using the below command:
+```
+yarn app -upgrade ${service_name} -initate ${path_to_new_service_def_file} [-autoFinalize]
+```
+
+e.g. To upgrade `my-sleeper` to sleep for *1200000* instead of *900000*, the user
+can upgrade the service to version 1.0.1. Below is the service definition for
+version 1.0.1 of sleeper-service:
+
+```
+{
+  "components" :
+    [
+      {
+        "name": "sleeper",
+        "version": "1.0.1",
+        "number_of_containers": 1,
+        "launch_command": "sleep 1200000",
+        "resource": {
+          "cpus": 1,
+          "memory": "256"
+        }
+      }
+    ]
+}
+```
+The command below initiates the upgrade to version 1.0.1.
+```
+yarn app -upgrade my-sleeper -initiate sleeper_v101.json
+```
+
+### Upgrade Instance
+User can upgrade a component instance using the below command:
+```
+yarn app -upgrade ${service_name} -instances ${comma_separated_list_of_instance_names}
+```
+e.g. The command below upgrades `sleeper-0` and `sleeper-1` instances of `my-service`:
+```
+yarn app -upgrade my-sleeper -instances sleeper-0,sleeper-1
+```
+
+### Upgrade Component
+User can upgrade a component, that is, all the instances of a component with
+one command:
+```
+yarn app -upgrade ${service_name} -components ${comma_separated_list_of_component_names}
+```
+e.g. The command below upgrades all the instances of `sleeper` component of `my-service`:
+```
+yarn app -ugrade my-sleeper -components sleeper
+```
+
+### Finalize Upgrade
+User must finalize the upgrade using the below command (since autoFinalize was not specified
during initiate):
+```
+yarn app -upgrade ${service_name} -finalize
+```
+e.g. The command below finalizes the upgrade of `my-sleeper`:
+```
+yarn app -upgrade my-sleeper -finalize
+```


---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-commits-help@hadoop.apache.org


Mime
View raw message