ambari-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chattopadhyay, Arka (Nokia - IN/Bangalore)" <arka.chattopadh...@nokia.com>
Subject Complete upgrade of HDP stack via Ambari REST API
Date Mon, 25 Sep 2017 11:53:04 GMT
Hi All,
Firstly I apologize for a long mail.
We are trying to completely automate (without any manual intervention) the upgrade of stack(HDP/any
other supported stack) through Ambari rest APIs. We searched in Internet and didn't find any
complete guide on this automation process, so we tried to come up with a guide for automation
for greater good of the Ambari community.
If similar things are tried and any feedback or experiences could be shared then it will be
of immense help for us.

Motivation:
In our case the automation is a part of bigger orchestration work. This automation will also
help in scenarios where the end user will not have any Ambari GUI access

References:
1. Ambari Rest API guide https://github.com/apache/ambari/blob/trunk/ambari-server/docs/api/v1/upgrades.md
2. Blog by Alejandro Fernandez https://community.hortonworks.com/articles/2473/rolling-upgrade-express-upgrade-in-ambari.html
3. Few trial and errors from our end.

Prerequisites/ Assumptions:
1. The stack is supported by Ambari for upgrade i.e. the stack should be manually upgradable
via Ambari.
2. The Ambari details like Ambari ip, port, login credentials, cluster name, etc. are available
from end user.
3. The stack details like repository, version, OS version, upgrade type, etc. are available
from end user.


Steps:
We are listing the Ambari rest APIs only which can be programed in any language. The user
inputs are marked with <> here. Following are steps and the respective Rest API/Curl
commands for upgrade:

1) Registering the Stack:
The first step is registering the new stack with Ambari. We used the following curl command
to do this and the corresponding json file format is:

               POST  http://<Ambari Ip>:<Ambari port>/api/v1/stacks/<New Stack
Name>/versions/<Stack Version>/repository_versions

               {
               "RepositoryVersions":
               {
                              "repository_version": "<New_Stack_Repository_Version>",
                              "display_name": "<Stack_Display_Name>"
               },
               "operating_systems": [
               {
                              "OperatingSystems":
                              {
                                             "os_type": "<OS_Type_and_Version>"
                              },
                              "repositories":
                              [
                              {
                                             "Repositories":
                                             {
                                                            "repo_id": "<Stack_Repository_ID>",
                                                            "repo_name": "<Stack_Repository_Name>",
                                                            "base_url": "<Stack_Repository_Base_URL>"
                                             }
                              },
                              {
                                             "Repositories":
                                             {
                                                            "repo_id": "<Stack_Utils_Repository_ID>",
                                                            "repo_name": "<Stack_Utils_Repository_Name>",
                                                            "base_url": "<Stack_Utils_Repository_Base_URL>"
                                             }
                              }
                              ]
               }
               ]
               }

We check the response of this command to confirm that the registration is successfull or not.
If the stack is already registered then this step can return error. We ignore this particular
error.

2) Installing the Stack:
The next step is installing the new stack on each host. We used the following curl command
to do this and the corresponding json file format is:

               POST http://<Ambari Ip>:<Ambari port>/api/v1/clusters/<Cluster
name>/stack_versions

               {
               "ClusterStackVersions":
               {
                              "stack": "<New_Stack_Name>",
                              "version": "<New_Stack_Version>",
                              "repository_version": "<New_Stack_Repository_Version>"
               }
               }

3) Check the status of Installation:
Since installation is a time-consuming process, so we continuously poll for the status of
the request (the request Id we got from the previous step) for failed or completed.
We used the following curl command to do this:

               GET http://<Ambari Ip>:<Ambari port>/api/v1/clusters/<Cluster
name>/requests/<Install request ID>

4) Upgrade of stack:
Once the installation is completed successfully, we supply the command to Ambari for upgrade
of stack.
We used the following curl command to do this and the corresponding json file format is:

               POST http://<Ambari Ip>:<Ambari port>/api/v1/clusters/<Cluster
name>/upgrades

               {
               "Upgrade":
               {
                              "repository_version": "<New_Stack_Repository_Version>",
                              "upgrade_type" : "<Upgrade_Type_Rolling_or_Express>"
               }
               }

5) Check status of Upgrade:
We continuously check for the upgrade item status for failed or holding.
During upgrade, there are multiple instances where the upgrade task will go for HOLDING (which
usually means Ambari need user intervention)
Example: During YARN upgrade, AMBARI will ask user if the YARN job queues are stopped/started.
When an upgrade_item goes to Holding status, we check the context of the upgrade_item and
take necessary action (Complete/Abort the upgrade_item) to automate the progress of upgrade.
We do this polling till the upgrade task goes on Holding for Finalize/Downgrade state. We
used the following curl command to do this:

               GET http://<Ambari Ip>:<Ambari port>/api/v1/clusters/<Cluster
name>/upgrades?upgrade_groups/upgrade_items/UpgradeItem/status=HOLDING

and check the context of holding upgrade item.

6) Downgrade or Finalize:
Once the upgrade is successful, Ambari upgrade goes to Holding state. In this step, the end
user have a choice to either Downgrade the stack OR Finalize the upgrade.
We take the user action on this and progress accordingly.

For Downgrade:
Once the user chooses to downgrade, we first abort the existing upgrade task which is in holding
state by issuing following command:

               POST http://<Ambari Ip>:<Ambari port>/api/v1/clusters/<Cluster
name>/upgrades/<Upgrade id>

               {
               "Upgrade":
               {
                              "request_status":"ABORTED",
                              "suspended":true
               }
               }

Then we issue a new downgrade request as follows:

               POST http://<Ambari Ip>:<Ambari port>/api/v1/clusters/<Cluster
name>/upgrades?downgrade=true

               {
               "Upgrade":
               {
                              "from_version": "<Old_Stack_Version>",
                              "repository_version": "<Old_Stack_Repository_Version>",
                              "direction": "DOWNGRADE",
                              "upgrade_type": "<Upgrade_Type_Rolling_or_Express>"
               }
               }

For Finalize:
Once the user chooses to finalize the stack, we complete the upgrade_item, which is in holding
state by following command:

               POST http://<Ambari Ip>:<Ambari port>/api/v1/clusters/<Cluster
name>/upgrades/<Upgrade id>/upgrade_groups/<Upgrade_group id>/upgrade_items/<Upgrade_item
id>

               {
               "UpgradeItem":
               {
                              "status":"COMPLETED"
               }
               }



Thanks and Regards
Arka Chattopadhyay

Mime
View raw message