Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============4196582426494669986=="
MIME-Version: 1.0
Subject: Re: Review Request 36592: RU - Magician Script to correct data
 inconsistencies, allow retrying repo installation,
 force finalize to versions, etc
From: "Alejandro Fernandez" <afernandez@hortonworks.com>
To: "Yusaku Sako" <yusaku@hortonworks.com>,
 "Nate Cole" <ncole@hortonworks.com>,
 "Sid Wagle" <swagle@hortonworks.com>,
 "Jonathan Hurley" <jhurley@hortonworks.com>
Cc: "Ambari" <dev@ambari.apache.org>,
 "Alejandro Fernandez" <afernandez@hortonworks.com>
Date: Mon, 10 Aug 2015 20:18:54 -0000
Message-ID: <20150810201854.1374.8912@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Alejandro Fernandez" <noreply@reviews.apache.org>
References: <20150720234241.10542.42873@reviews.apache.org>
In-Reply-To: <20150720234241.10542.42873@reviews.apache.org>
Reply-To: "Alejandro Fernandez" <afernandez@hortonworks.com>

--===============4196582426494669986==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36592/
-----------------------------------------------------------

(Updated Aug. 10, 2015, 8:18 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, Sid Wagle, and Yusaku Sako.


Changes
-------

After discussions with Apache Legal time, it is ok to submit this to trunk.
The script can be licensed under ASF v2 because the psycopg2 library, which uses LGPL, falls under the "optional" clause.
LGPL v2.1 is meant to handle this case specifically where
"
simply compiling software to “make use” of the library does not in itself cause the compiled work to be a “work based on the library.”
"

To connect to MySQL, it uses PyMySQL, which uses MIT license.


Bugs: AMBARI-12455
    https://issues.apache.org/jira/browse/AMBARI-12455


Repository: ambari


Description
-------

Support has identified the need to come up with a script to fix any mismatches in the database, or identify problems during Rolling Upgrade.
This can be a simple Python script that,

- On a newly installed cluster, ensures that there is at least one cluster version whose state is CURRENT. If not, will advise the user to restart services.
- If the user has Registered and Installed repos, check that each one has a unique version and display name. Further, if any are stuck in an INSTALLING state, will let the user take three potential actions: leave as is, force to INSTALLED, force to INSTALL_FAILED.
- If the user has Registered and Installed repos, and one cluster_version is already in an UPGRADING state, perhaps because hdp-select changed the symlinks and a component was restarted, or the user inadvertently started a manual upgrade, will allow the user to force it back to INSTALLED.
- If the user in the in the middle of an upgrade, and they want to force one of the versions are CURRENT, it will update all DB records accordingly, and set the previously CURRENT version to INSTALLED.
For now, this will support Ambari 2.0.0 and higher, and MySQL, and Postgres.


Diffs (updated)
-----

  contrib/ru_magician/README.md PRE-CREATION 
  contrib/ru_magician/lgpl_license.txt PRE-CREATION 
  contrib/ru_magician/mit_license.txt PRE-CREATION 
  contrib/ru_magician/ru_magician.py PRE-CREATION 
  pom.xml 5b5c860 

Diff: https://reviews.apache.org/r/36592/diff/


Testing
-------

Tested on Ambari 2.1.0 with Postgres, and Ambari 2.0.1 on MySQL (using local and external DB), all of the scenarios above.
I still have to do more thorough testing in the cases of Finalize, but this is a good preliminary patch to start getting some feedback.

I could have used SQLAlchemy as my ORM, but I really wanted a standalone script that could be a quick v1, so I went for raw SQL that works on both Postgres & MySQL.


Thanks,

Alejandro Fernandez


--===============4196582426494669986==--