ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitry Lysnichenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-20875) Removing A Service Causes DB Verification To Produce Warnings
Date Wed, 03 May 2017 13:36:04 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-20875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994863#comment-15994863
] 

Dmitry Lysnichenko commented on AMBARI-20875:
---------------------------------------------

Committed a correct patch for trunk
To https://git-wip-us.apache.org/repos/asf/ambari.git
   7d24bfc..f942bb2  trunk -> trunk


> Removing A Service Causes DB Verification To Produce Warnings
> -------------------------------------------------------------
>
>                 Key: AMBARI-20875
>                 URL: https://issues.apache.org/jira/browse/AMBARI-20875
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.4.2
>            Reporter: Dmitry Lysnichenko
>            Assignee: Dmitry Lysnichenko
>             Fix For: 2.5.2
>
>         Attachments: AMBARI-20875.patch
>
>
> When removing a service, the configurations for that service are kept for historical
purposes, but their various associations in the database are removed (specifically, the {{serviceconfigmapping}}
relationships).
> After removing a service, the orphaned configurations now cause a warning to be displayed
on Ambari Server startup:
> {noformat}
> 2017-04-06 17:15:24,003  WARN - You have config(s): ranger-storm-policymgr-ssl-version1467149286586,atlas-env-version1471883877194,falcon-env-version1467044148480,storm-site-version1467149286586,storm-site-version1474944944095,ranger-storm-plugin-properties-version1467149286586,hana_hadoop-env-version1476989318735,hana_hadoop-env-version1468951412523,hanaes-site-version1475773173499,hanaes-site-version1477639131416,atlas-env-version1471880496396,falcon-startup.properties-version1474944962583,ranger-storm-security-version1467149286586,falcon-env-version1474944962517,application-properties-version1471883877194,hanaes-site-version1468951412523,application-properties-version1471992143777,application-properties-version1471880496396,hana_hadoop-env-version1475790068354,hana_hadoop-env-version1477639131416,falcon-runtime.properties-version1467044148480,atlas-env-version1471992143777,hana_hadoop-env-version1475773173499,storm-env-version1467149286586,hanaes-site-version1475790068354,hanaes-site-version1476902714170,atlas-env-version1471883827584,hana_hadoop-env-version1477695406433,hanaes-site-version1476989583427,falcon-log4j-version1,falcon-env-version1474944962457,hanaes-site-version1468959251565,falcon-client.properties-version1,atlas-env-version1471993347065,falcon-startup.properties-version1467044148480,storm-cluster-log4j-version1467149286586,hanaes-site-version1472285532383,hana_hadoop-env-version1477695089738,hana_hadoop-env-version1468959251565,hana_hadoop-env-version1476989821279,atlas-log4j-version1,storm-site-version1467612840864,storm-worker-log4j-version1467149286586,ranger-storm-audit-version1467149286586,application-properties-version1471993347065,application-properties-version1471883827584,hana_hadoop-env-version1477695579450
that is(are) not mapped (in serviceconfigmapping table) to any service!
> {noformat}
> These orphaned configurations have entries in both {{clusterconfig}} and {{clusterconfigmapping}}
but are otherwise not references anywhere. They don't hurt anything, but do trigger this warning
since we can't determine if they _should_ have mappings in {{serviceconfigmapping}}.
> A few options:
> - When removing a service, remove configurations as well, leaving no orphans. Some would
argue that this is a bad move since re-adding the service later would allow you to see the
old configurations. I do not believe this is true since the old configurations are never associated
with the new service's {{serviceconfig}} or {{serviceconfigmapping}}.
> - Make the warning smarter somehow to ignore these since it's expected they are orphaned.
> -- Somehow determine the service which should own the config and see if it exists in
the cluster?
> -- Add a new column to {{clusterconfig}} to mark it as deleted?
> To clean these warnings, we had to:
> {code}
> CREATE TEMPORARY TABLE IF NOT EXISTS orphaned_configs AS
> (SELECT
> cc.config_id,
> cc.type_name,
> cc.version_tag
> FROM clusterconfig cc, clusterconfigmapping ccm
> WHERE cc.config_id NOT IN (SELECT
> scm.config_id
> FROM serviceconfigmapping scm)
> AND cc.type_name != 'cluster-env'
> AND cc.type_name = ccm.type_name
> AND cc.version_tag = ccm.version_tag);
> DELETE FROM clusterconfigmapping
> WHERE EXISTS
> (SELECT 1 FROM orphaned_configs
> WHERE clusterconfigmapping.type_name = orphaned_configs.type_name AND clusterconfigmapping.version_tag
= orphaned_configs.version_tag);
> DELETE FROM clusterconfig WHERE clusterconfig.config_id IN (SELECT config_id FROM orphaned_configs);
> SELECT * FROM orphaned_configs;
> DROP TABLE orphaned_configs;
> {code}
> I've considered advanced heuristics based on service metainfo with config dependencies,
and service config mappings. But this approach may be unreliable when given inaccurate service
metainfo between stack upgrades. Also, this approach is likely to delay server start on clusters
with thousands of configs.
> So the solution Add a new column to clusterconfig to mark it as deleted. So warning will
not be generated for such cluster configs.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message