ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 32557: Full Delete of Host : Switch host-related tables to use host_id instead of host_name column for hoststate, hostcomponentstate, hostcomponentdesiredstate
Date Thu, 02 Apr 2015 01:17:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/32557/
-----------------------------------------------------------

(Updated April 2, 2015, 1:16 a.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Sid Wagle.


Changes
-------

Rebased patch. Agreed to fix 1.5 - 2.0 UpgradeCatalogs in a separate Jira


Bugs: AMBARI-10168
    https://issues.apache.org/jira/browse/AMBARI-10168


Repository: ambari


Description
-------

See p;arent Jira AMBARI-10167

Deleting a host from the UI only removes it from the mapping to a cluster.
The host still exists at the hosts endpoint, and all of the historic data maintains intact;
config overrides, config groups, host state, host version, host role command logs, etc.

This causes two problems.
1. If a host is removed from a cluster and its underlying environment changes (OS, IP, network
card, etc), and is re-added to the cluster, then it still maintains the old information that
is incorrect.
2. If host x dies, and is removed from the cluster, there is no way for a new host to take
over its hostname.
Expectation in Ambari 2.1.0
Deleting a host should be clean, i.e., it should delete all configuration and historic records.
Because renaming a host requires a more complicated workflow, so it will have to be done manually
by the user (see instructions in A1).
Lastly, Ambari will require ambari-agent to be stopped before the host can be deleted.


Tables:
10 tables have foreign keys to the host_name in the hosts table:
1. hostcomponentstate         (fixed in this patch)
2. hostcomponentdesiredstate  (fixed in this patch)
3. hoststate                  (fixed in this patch)
4. host_version               (future patch)
5. host_role_command          (future patch)
6. request_operation_level    (future patch)
7. hostconfigmapping          (future patch)
8. configgrouphostmapping     (future patch)
9. kerberos_principal_host    (future patch)
10. clusterhostmapping        (already fixed)


Diffs (updated)
-----

  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAO.java
8e92877 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostDAO.java 0fb9c59 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntity.java
e0f1e9e 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentDesiredStateEntityPK.java
fc92858 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntity.java
0a31b90 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostComponentStateEntityPK.java
bc103a1 
  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostEntity.java 4df5f39

  ambari-server/src/main/java/org/apache/ambari/server/orm/entities/HostStateEntity.java 5d5a1e8

  ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java 1bd2814

  ambari-server/src/main/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostImpl.java
7975f4e 
  ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
1f3c09a 
  ambari-server/src/main/java/org/apache/ambari/server/upgrade/SchemaUpgradeHelper.java 3691af2

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog.java fcf4f51

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog150.java dd8e33e

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog151.java afe27b5

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog160.java a1f11d8

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog161.java 4ab0cb6

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog170.java 98ac89f

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog200.java d3b062a

  ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog210.java 23ed67d

  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 7593e07 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 0562bc6 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 44fde5f 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 74ea215 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 2a7d3f9 
  ambari-server/src/test/java/org/apache/ambari/server/orm/dao/HostComponentDesiredStateDAOTest.java
7b34827 
  ambari-server/src/test/java/org/apache/ambari/server/state/ServiceComponentTest.java 3093c2f

  ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClustersTest.java c2ab35f

  ambari-server/src/test/java/org/apache/ambari/server/state/svccomphost/ServiceComponentHostTest.java
ba19154 
  ambari-server/src/test/java/org/apache/ambari/server/upgrade/SectionDDL.java PRE-CREATION

  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog150Test.java
af6721c 
  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog170Test.java
6fa3e0a 
  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog200Test.java
9c39749 
  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog210Test.java
47cadf3 
  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogHelper.java b5db360

  ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalogTest.java b0c8fd9


Diff: https://reviews.apache.org/r/32557/diff/


Testing
-------

Unit tests passed,
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ....................................... SUCCESS [4.737s]
[INFO] Apache Ambari Project POM ......................... SUCCESS [0.031s]
[INFO] Ambari Web ........................................ SUCCESS [38.862s]
[INFO] Ambari Views ...................................... SUCCESS [2.078s]
[INFO] Ambari Admin View ................................. SUCCESS [9.289s]
[INFO] Ambari Metrics Common ............................. SUCCESS [1.487s]
[INFO] Ambari Server ..................................... SUCCESS [34:09.719s]
[INFO] Ambari Agent ...................................... SUCCESS [21.818s]
[INFO] Ambari Client ..................................... SUCCESS [0.033s]
[INFO] Ambari Python Client .............................. SUCCESS [0.254s]
[INFO] Ambari Groovy Client .............................. SUCCESS [9.834s]
[INFO] Ambari Shell ...................................... SUCCESS [0.032s]
[INFO] Ambari Python Shell ............................... SUCCESS [0.044s]
[INFO] Ambari Groovy Shell ............................... SUCCESS [6.661s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 35:45.473s
[INFO] Finished at: Thu Mar 26 17:39:24 PDT 2015
[INFO] Final Memory: 66M/500M
[INFO] ------------------------------------------------------------------------

I tested this by making the DB changes, copying my jar, then hitting the dashboard, API, and
restarting services on a cluster.


-- For Postgres
--- Fix hoststate
SELECT COUNT(*) FROM hoststate;
ALTER TABLE hoststate ADD COLUMN host_id BIGINT NULL;
UPDATE hoststate t SET host_id = (SELECT host_id FROM hosts h WHERE h.host_name = t.host_name)
WHERE t.host_id IS NULL AND t.host_name IS NOT NULL;
ALTER TABLE hoststate ALTER COLUMN host_id SET NOT NULL;
ALTER TABLE hoststate ADD CONSTRAINT FK_hoststate_host_id FOREIGN KEY (host_id) REFERENCES
hosts (host_id);
ALTER TABLE hoststate DROP COLUMN host_name;
ALTER TABLE hoststate ADD CONSTRAINT hoststate_pkey PRIMARY KEY (host_id);

-- Fix hostcomponentstate
SELECT COUNT(*) FROM hostcomponentstate;
ALTER TABLE hostcomponentstate ADD COLUMN host_id BIGINT NULL;
UPDATE hostcomponentstate t SET host_id = (SELECT host_id FROM hosts h WHERE h.host_name =
t.host_name) WHERE t.host_id IS NULL AND t.host_name IS NOT NULL;
ALTER TABLE hostcomponentstate ALTER COLUMN host_id SET NOT NULL;
ALTER TABLE hostcomponentstate ADD CONSTRAINT FK_hostcomponentstate_host_id FOREIGN KEY (host_id)
REFERENCES hosts (host_id);
ALTER TABLE hostcomponentstate DROP COLUMN host_name;
ALTER TABLE hostcomponentstate ADD CONSTRAINT hostcomponentstate_pkey PRIMARY KEY (cluster_id,
component_name, host_id, service_name);

-- Fix hostcomponentdesiredstate
SELECT COUNT(*) FROM hostcomponentdesiredstate;
ALTER TABLE hostcomponentdesiredstate ADD COLUMN host_id BIGINT NULL;
UPDATE hostcomponentdesiredstate t SET host_id = (SELECT host_id FROM hosts h WHERE h.host_name
= t.host_name) WHERE t.host_id IS NULL AND t.host_name IS NOT NULL;
ALTER TABLE hostcomponentdesiredstate ALTER COLUMN host_id SET NOT NULL;
ALTER TABLE hostcomponentdesiredstate ADD CONSTRAINT FK_hostcomponentdesiredstate_host_id
FOREIGN KEY (host_id) REFERENCES hosts (host_id);
ALTER TABLE hostcomponentdesiredstate DROP COLUMN host_name;
ALTER TABLE hostcomponentdesiredstate ADD CONSTRAINT hostcomponentdesiredstate_pkey PRIMARY
KEY (cluster_id, component_name, host_id, service_name);


Thanks,

Alejandro Fernandez


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message