Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Content-Type: multipart/alternative;
 boundary="===============4534093785503232987=="
MIME-Version: 1.0
Subject: Re: Review Request 32192: Full Delete of Host : Add host_id column to
 clusterhostmapping, and create initial UpgradeCatalog210
From: "Alejandro Fernandez" <afernandez@hortonworks.com>
To: "Sid Wagle" <swagle@hortonworks.com>, "Nate Cole" <ncole@hortonworks.com>,
 "Ivan Kozlov" <eldorado@ukrdc.net>,
 "Jonathan Hurley" <jhurley@hortonworks.com>
Cc: "Ambari" <dev@ambari.apache.org>,
 "Alejandro Fernandez" <afernandez@hortonworks.com>
Date: Tue, 24 Mar 2015 00:03:48 -0000
Message-ID: <20150324000348.32590.32068@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: "Alejandro Fernandez" <noreply@reviews.apache.org>
References: <20150323234141.32590.56633@reviews.apache.org>
In-Reply-To: <20150323234141.32590.56633@reviews.apache.org>
Reply-To: "Alejandro Fernandez" <afernandez@hortonworks.com>

--===============4534093785503232987==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32192/
-----------------------------------------------------------

(Updated March 24, 2015, 12:03 a.m.)


Review request for Ambari, Ivan Kozlov, Jonathan Hurley, Nate Cole, and Sid Wagle.


Changes
-------

Rebased


Bugs: AMBARI-10190
    https://issues.apache.org/jira/browse/AMBARI-10190


Repository: ambari


Description
-------

This is going to be a really big undertaking, so I'm going to do it in small chunks, otherwise, the code reviews and rebases are going to be massive.

* Add the column id to the hosts table, and populate it with auto-increment values
* Temporarily remove and recreate all FKs to the host_name column, in order to change the PK of the hosts table from the host_name to the id column
* Add the column host_id to the clusterhostmapping table and populate it
* Drop the host_name column from the clusterhostmapping table
* Create the initial UpgradeCatalog210.java file and unit test


Parent Jira is https://issues.apache.org/jira/browse/AMBARI-10167


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 8833148 
  ambari-server/src/main/java/org/apache/ambari/server/api/query/JpaPredicateVisitor.java afbb3e2 
  ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java 7248459 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java 5435982 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostDAO.java 35c795b 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java 3255e58 
  ambari-server/src/main/java/org/apache/ambari/server/state/Alert.java 3211cfc 
  ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java 5968b2f 
  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java PRE-CREATION 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9ff62df 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 7d62aee 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 78a263f 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql a06f1d2 
  ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java 0f98a3d 
  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog210Test.java PRE-CREATION 

Diff: https://reviews.apache.org/r/32192/diff/


Testing
-------

Local unit tests UpgradeCatalog*.java and UpgradeTest.java passed.
Waiting for full set of unit test results.


I applied the following migration steps manually, then copied ambari-server jar, restarted the server, and was still able to access the hosts on the cluster via the UI and API.

```
-- Changes to the hosts table
ALTER TABLE hosts ADD CONSTRAINT UQ_hosts_host_name UNIQUE (host_name); -- This will persist, even after the PK changes.

-- Must first create with allowing null
ALTER table hosts add column id BIGINT NULL;

SELECT COUNT(*) FROM hosts;

CREATE SEQUENCE host_id_seq;
UPDATE hosts SET id=nextval('host_id_seq');
SELECT currval('host_id_seq');  --should equal the same number as the count(*)


INSERT INTO ambari_sequences (sequence_name, sequence_value) VALUES ('host_id_seq', (SELECT currval('host_id_seq')));
SELECT * FROM ambari_sequences WHERE sequence_name = 'host_id_seq';

DROP sequence host_id_seq;

ALTER TABLE hosts ALTER column id SET NOT NULL;

-- Drop and re-create 8 FKs
ALTER TABLE hostcomponentdesiredstate DROP CONSTRAINT hstcmponentdesiredstatehstname;
ALTER TABLE hostcomponentstate DROP CONSTRAINT hostcomponentstate_host_name;
ALTER TABLE hoststate DROP CONSTRAINT FK_hoststate_host_name;
ALTER TABLE host_version DROP CONSTRAINT FK_host_version_host_name;
ALTER TABLE host_role_command DROP CONSTRAINT FK_host_role_command_host_name;
ALTER TABLE hostconfigmapping DROP CONSTRAINT FK_hostconfmapping_host_name;
ALTER TABLE configgrouphostmapping DROP CONSTRAINT FK_cghm_hname;
ALTER TABLE kerberos_principal_host DROP CONSTRAINT FK_krb_pr_host_hostname;

-- This is a bogus FK that does not need to be added. It only exists in Postgres Embedded.
ALTER TABLE clusterhostmapping DROP CONSTRAINT clusterhostmapping_host_name;

-- Drop the PK, and recreate it on the id instead
ALTER TABLE hosts DROP CONSTRAINT hosts_pkey;
ALTER TABLE hosts ADD CONSTRAINT PK_hosts_id PRIMARY KEY (id);

ALTER TABLE hostcomponentdesiredstate ADD CONSTRAINT hstcmponentdesiredstatehstname FOREIGN KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE hostcomponentstate ADD CONSTRAINT hostcomponentstate_host_name FOREIGN KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE hoststate ADD CONSTRAINT FK_hoststate_host_name FOREIGN KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE host_version ADD CONSTRAINT FK_host_version_host_name FOREIGN KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE host_role_command ADD CONSTRAINT FK_host_role_command_host_name FOREIGN KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE hostconfigmapping ADD CONSTRAINT FK_hostconfmapping_host_name FOREIGN KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE configgrouphostmapping ADD CONSTRAINT FK_cghm_hname FOREIGN KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE kerberos_principal_host ADD CONSTRAINT FK_krb_pr_host_hostname FOREIGN KEY (host_name) REFERENCES hosts (host_name);


-- Fix the clusterhostmapping
SELECT COUNT(*) FROM clusterhostmapping;
ALTER TABLE clusterhostmapping ADD COLUMN host_id BIGINT NULL;
UPDATE clusterhostmapping chm SET host_id = (SELECT id FROM hosts h WHERE h.host_name = chm.host_name) WHERE chm.host_id IS NULL AND chm.host_name IS NOT NULL;
ALTER TABLE clusterhostmapping ADD CONSTRAINT FK_clusterhostmapping_hosts_id FOREIGN KEY (host_id) REFERENCES hosts (id);

ALTER TABLE clusterhostmapping DROP COLUMN host_name;
```


Thanks,

Alejandro Fernandez


--===============4534093785503232987==--