ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 32192: Full Delete of Host : Add host_id column to clusterhostmapping, and create initial UpgradeCatalog210
Date Mon, 23 Mar 2015 23:39:48 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32192/
-----------------------------------------------------------

(Updated March 23, 2015, 11:39 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle.


Summary (updated)
-----------------

Full Delete of Host : Add host_id column to clusterhostmapping, and create initial UpgradeCatalog210


Bugs: AMBARI-10190
    https://issues.apache.org/jira/browse/AMBARI-10190


Repository: ambari


Description (updated)
-------

This is going to be a really big undertaking, so I'm going to do it in small chunks, otherwise,
the code reviews and rebases are going to be massive.

* Add the column id to the hosts table, and populate it with auto-increment values
* Temporarily remove and recreate all FKs to the host_name column, in order to change the
PK of the hosts table from the host_name to the id column
* Add the column host_id to the clusterhostmapping table and populate it
* Drop the host_name column from the clusterhostmapping table
* Create the initial UpgradeCatalog210.java file and unit test


Parent Jira is https://issues.apache.org/jira/browse/AMBARI-10167


Diffs
-----

  ambari-server/src/main/java/org/apache/ambari/server/agent/HeartBeatHandler.java 8833148

  ambari-server/src/main/java/org/apache/ambari/server/api/query/JpaPredicateVisitor.java
afbb3e2 
  ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertResourceProvider.java
a25cba0 
  ambari-server/src/main/java/org/apache/ambari/server/events/listeners/alerts/AlertReceivedListener.java
7248459 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/AlertsDAO.java 5435982 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostDAO.java 35c795b 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertCurrentEntity.java
66aa119 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertHistoryEntity.java
8e96aca 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java 3255e58

  ambari-server/src/main/java/org/apache/ambari/server/state/Alert.java 3211cfc 
  ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
974dcdf 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 9ff62df 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 7d62aee 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 78a263f 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql a06f1d2 
  ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertResourceProviderTest.java
5459a8c 
  ambari-server/src/test/java/org/apache/ambari/server/orm/AlertDaoHelper.java 674c092 
  ambari-server/src/test/java/org/apache/ambari/server/orm/OrmTestHelper.java b9fc424 
  ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertDispatchDAOTest.java a267043

  ambari-server/src/test/java/org/apache/ambari/server/orm/dao/AlertsDAOTest.java 9c8ea7d

  ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertEventPublisherTest.java
10535e6 
  ambari-server/src/test/java/org/apache/ambari/server/state/alerts/AlertReceivedListenerTest.java
0f98a3d 
  ambari-server/src/test/java/org/apache/ambari/server/state/cluster/AlertDataManagerTest.java
acf7911 
  ambari-server/src/test/java/org/apache/ambari/server/state/services/AlertNoticeDispatchServiceTest.java
3f09db0 

Diff: https://reviews.apache.org/r/32192/diff/


Testing (updated)
-------

Local unit tests UpgradeCatalog*.java and UpgradeTest.java passed.
Waiting for full set of unit test results.


I applied the following migration steps manually, then copied ambari-server jar, restarted
the server, and was still able to access the hosts on the cluster via the UI and API.

```
-- Changes to the hosts table
ALTER TABLE hosts ADD CONSTRAINT UQ_hosts_host_name UNIQUE (host_name); -- This will persist,
even after the PK changes.

-- Must first create with allowing null
ALTER table hosts add column id BIGINT NULL;

SELECT COUNT(*) FROM hosts;

CREATE SEQUENCE host_id_seq;
UPDATE hosts SET id=nextval('host_id_seq');
SELECT currval('host_id_seq');  --should equal the same number as the count(*)


INSERT INTO ambari_sequences (sequence_name, sequence_value) VALUES ('host_id_seq', (SELECT
currval('host_id_seq')));
SELECT * FROM ambari_sequences WHERE sequence_name = 'host_id_seq';

DROP sequence host_id_seq;

ALTER TABLE hosts ALTER column id SET NOT NULL;

-- Drop and re-create 8 FKs
ALTER TABLE hostcomponentdesiredstate DROP CONSTRAINT hstcmponentdesiredstatehstname;
ALTER TABLE hostcomponentstate DROP CONSTRAINT hostcomponentstate_host_name;
ALTER TABLE hoststate DROP CONSTRAINT FK_hoststate_host_name;
ALTER TABLE host_version DROP CONSTRAINT FK_host_version_host_name;
ALTER TABLE host_role_command DROP CONSTRAINT FK_host_role_command_host_name;
ALTER TABLE hostconfigmapping DROP CONSTRAINT FK_hostconfmapping_host_name;
ALTER TABLE configgrouphostmapping DROP CONSTRAINT FK_cghm_hname;
ALTER TABLE kerberos_principal_host DROP CONSTRAINT FK_krb_pr_host_hostname;

-- This is a bogus FK that does not need to be added. It only exists in Postgres Embedded.
ALTER TABLE clusterhostmapping DROP CONSTRAINT clusterhostmapping_host_name;

-- Drop the PK, and recreate it on the id instead
ALTER TABLE hosts DROP CONSTRAINT hosts_pkey;
ALTER TABLE hosts ADD CONSTRAINT PK_hosts_id PRIMARY KEY (id);

ALTER TABLE hostcomponentdesiredstate ADD CONSTRAINT hstcmponentdesiredstatehstname FOREIGN
KEY (host_name) REFERENCES hosts (host_name);
ALTER TABLE hostcomponentstate ADD CONSTRAINT hostcomponentstate_host_name FOREIGN KEY (host_name)
REFERENCES hosts (host_name);
ALTER TABLE hoststate ADD CONSTRAINT FK_hoststate_host_name FOREIGN KEY (host_name) REFERENCES
hosts (host_name);
ALTER TABLE host_version ADD CONSTRAINT FK_host_version_host_name FOREIGN KEY (host_name)
REFERENCES hosts (host_name);
ALTER TABLE host_role_command ADD CONSTRAINT FK_host_role_command_host_name FOREIGN KEY (host_name)
REFERENCES hosts (host_name);
ALTER TABLE hostconfigmapping ADD CONSTRAINT FK_hostconfmapping_host_name FOREIGN KEY (host_name)
REFERENCES hosts (host_name);
ALTER TABLE configgrouphostmapping ADD CONSTRAINT FK_cghm_hname FOREIGN KEY (host_name) REFERENCES
hosts (host_name);
ALTER TABLE kerberos_principal_host ADD CONSTRAINT FK_krb_pr_host_hostname FOREIGN KEY (host_name)
REFERENCES hosts (host_name);


-- Fix the clusterhostmapping
SELECT COUNT(*) FROM clusterhostmapping;
ALTER TABLE clusterhostmapping ADD COLUMN host_id BIGINT NULL;
UPDATE clusterhostmapping chm SET host_id = (SELECT id FROM hosts h WHERE h.host_name = chm.host_name)
WHERE chm.host_id IS NULL AND chm.host_name IS NOT NULL;
ALTER TABLE clusterhostmapping ADD CONSTRAINT FK_clusterhostmapping_hosts_id FOREIGN KEY (host_id)
REFERENCES hosts (id);

ALTER TABLE clusterhostmapping DROP COLUMN host_name;
```


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message