Mailing-List: contact issues-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Date: Wed, 3 May 2017 12:43:04 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: issues@ambari.apache.org
Message-ID: <JIRA.13067438.1493309425000.114589.1493815384293@Atlassian.JIRA>
In-Reply-To: <JIRA.13067438.1493309425000@Atlassian.JIRA>
References: <JIRA.13067438.1493309425000@Atlassian.JIRA> <JIRA.13067438.1493309425194@jira-lw-us.apache.org>
Subject: [jira] [Commented] (AMBARI-20875) Removing A Service Causes DB
 Verification To Produce Warnings
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
archived-at: Wed, 03 May 2017 12:43:09 -0000


    [ https://issues.apache.org/jira/browse/AMBARI-20875?page=3Dcom.atlassi=
an.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15=
994777#comment-15994777 ]=20

Hudson commented on AMBARI-20875:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7414 (See [https:=
//builds.apache.org/job/Ambari-trunk-Commit/7414/])
AMBARI-20875. Removing A Service Causes DB Verification To Produce (dlysnic=
henko: [http://git-wip-us.apache.org/repos/asf?p=3Dambari.git&a=3Dcommit&h=
=3Db061b1be2dca70d73ff82b873a83aad003c18a1a])
* (edit) ambari-server/src/main/java/org/apache/ambari/server/state/Service=
Impl.java
* (edit) ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql
* (edit) ambari-server/src/main/java/org/apache/ambari/server/upgrade/Schem=
aUpgradeHelper.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/checks/Databa=
seConsistencyCheckHelper.java
* (edit) ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql
* (edit) ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql
* (edit) ambari-server/src/main/java/org/apache/ambari/server/orm/entities/=
ClusterConfigEntity.java
* (edit) ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql
* (edit) ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql
* (edit) ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql


> Removing A Service Causes DB Verification To Produce Warnings
> -------------------------------------------------------------
>
>                 Key: AMBARI-20875
>                 URL: https://issues.apache.org/jira/browse/AMBARI-20875
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.4.2
>            Reporter: Dmitry Lysnichenko
>            Assignee: Dmitry Lysnichenko
>             Fix For: 2.5.2
>
>         Attachments: AMBARI-20875.patch
>
>
> When removing a service, the configurations for that service are kept for=
 historical purposes, but their various associations in the database are re=
moved (specifically, the {{serviceconfigmapping}} relationships).
> After removing a service, the orphaned configurations now cause a warning=
 to be displayed on Ambari Server startup:
> {noformat}
> 2017-04-06 17:15:24,003  WARN - You have config(s): ranger-storm-policymg=
r-ssl-version1467149286586,atlas-env-version1471883877194,falcon-env-versio=
n1467044148480,storm-site-version1467149286586,storm-site-version1474944944=
095,ranger-storm-plugin-properties-version1467149286586,hana_hadoop-env-ver=
sion1476989318735,hana_hadoop-env-version1468951412523,hanaes-site-version1=
475773173499,hanaes-site-version1477639131416,atlas-env-version147188049639=
6,falcon-startup.properties-version1474944962583,ranger-storm-security-vers=
ion1467149286586,falcon-env-version1474944962517,application-properties-ver=
sion1471883877194,hanaes-site-version1468951412523,application-properties-v=
ersion1471992143777,application-properties-version1471880496396,hana_hadoop=
-env-version1475790068354,hana_hadoop-env-version1477639131416,falcon-runti=
me.properties-version1467044148480,atlas-env-version1471992143777,hana_hado=
op-env-version1475773173499,storm-env-version1467149286586,hanaes-site-vers=
ion1475790068354,hanaes-site-version1476902714170,atlas-env-version14718838=
27584,hana_hadoop-env-version1477695406433,hanaes-site-version1476989583427=
,falcon-log4j-version1,falcon-env-version1474944962457,hanaes-site-version1=
468959251565,falcon-client.properties-version1,atlas-env-version14719933470=
65,falcon-startup.properties-version1467044148480,storm-cluster-log4j-versi=
on1467149286586,hanaes-site-version1472285532383,hana_hadoop-env-version147=
7695089738,hana_hadoop-env-version1468959251565,hana_hadoop-env-version1476=
989821279,atlas-log4j-version1,storm-site-version1467612840864,storm-worker=
-log4j-version1467149286586,ranger-storm-audit-version1467149286586,applica=
tion-properties-version1471993347065,application-properties-version14718838=
27584,hana_hadoop-env-version1477695579450 that is(are) not mapped (in serv=
iceconfigmapping table) to any service!
> {noformat}
> These orphaned configurations have entries in both {{clusterconfig}} and =
{{clusterconfigmapping}} but are otherwise not references anywhere. They do=
n't hurt anything, but do trigger this warning since we can't determine if =
they _should_ have mappings in {{serviceconfigmapping}}.
> A few options:
> - When removing a service, remove configurations as well, leaving no orph=
ans. Some would argue that this is a bad move since re-adding the service l=
ater would allow you to see the old configurations. I do not believe this i=
s true since the old configurations are never associated with the new servi=
ce's {{serviceconfig}} or {{serviceconfigmapping}}.
> - Make the warning smarter somehow to ignore these since it's expected th=
ey are orphaned.
> -- Somehow determine the service which should own the config and see if i=
t exists in the cluster?
> -- Add a new column to {{clusterconfig}} to mark it as deleted?
> To clean these warnings, we had to:
> {code}
> CREATE TEMPORARY TABLE IF NOT EXISTS orphaned_configs AS
> (SELECT
> cc.config_id,
> cc.type_name,
> cc.version_tag
> FROM clusterconfig cc, clusterconfigmapping ccm
> WHERE cc.config_id NOT IN (SELECT
> scm.config_id
> FROM serviceconfigmapping scm)
> AND cc.type_name !=3D 'cluster-env'
> AND cc.type_name =3D ccm.type_name
> AND cc.version_tag =3D ccm.version_tag);
> DELETE FROM clusterconfigmapping
> WHERE EXISTS
> (SELECT 1 FROM orphaned_configs
> WHERE clusterconfigmapping.type_name =3D orphaned_configs.type_name AND c=
lusterconfigmapping.version_tag =3D orphaned_configs.version_tag);
> DELETE FROM clusterconfig WHERE clusterconfig.config_id IN (SELECT config=
_id FROM orphaned_configs);
> SELECT * FROM orphaned_configs;
> DROP TABLE orphaned_configs;
> {code}
> I've considered advanced heuristics based on service metainfo with config=
 dependencies, and service config mappings. But this approach may be unreli=
able when given inaccurate service metainfo between stack upgrades. Also, t=
his approach is likely to delay server start on clusters with thousands of =
configs.
> So the solution Add a new column to clusterconfig to mark it as deleted. =
So warning will not be generated for such cluster configs.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)