ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-19130) Downgrade Can Create Multiple Mappings For Latest Configs
Date Thu, 08 Dec 2016 23:20:58 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-19130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733699#comment-15733699
] 

Hudson commented on AMBARI-19130:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #6193 (See [https://builds.apache.org/job/Ambari-trunk-Commit/6193/])
AMBARI-19130 - Downgrade Can Create Multiple Mappings For Latest Configs (jhurley: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=72586a1ed95728dff642758545cc9676fb77aa9e])
* (edit) ambari-server/src/main/java/org/apache/ambari/server/orm/entities/ClusterConfigMappingEntity.java
* (edit) ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
* (edit) ambari-server/src/test/java/org/apache/ambari/server/orm/dao/ServiceConfigDAOTest.java
* (edit) ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java


> Downgrade Can Create Multiple Mappings For Latest Configs
> ---------------------------------------------------------
>
>                 Key: AMBARI-19130
>                 URL: https://issues.apache.org/jira/browse/AMBARI-19130
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.2.0
>            Reporter: Jonathan Hurley
>            Assignee: Jonathan Hurley
>            Priority: Critical
>             Fix For: 2.5.0
>
>         Attachments: AMBARI-19130.patch
>
>
> During a downgrade which crosses major stack versions (such as from HDP 2.y to HDP 2.x),
Ambari attempts to set the latest configurations from HDP 2.x as "current". However, after
the downgrade succeeds, the database fails with the following consistency check:
> {code}
> ERROR - You have config(s), in cluster c1, that is(are) selected more than once in clusterconfigmapping
table: ranger-storm-plugin-properties,mapred-env,webhcat-env,hive-exec-log4j,hive-log4j,webhcat-log4j,storm-env,yarn-log4j,hcat-env
> {code}
> The problem is actually not with {{clusterconfig}} but with the state of the {{clusterconfigmapping}}
table _before_ upgrade. Apparently, it is possible to have multiple mappings for the same
config. We match on the config type and tag (such as hdfs-site / version1234566789). There
should never, EVER be duplicate mappings for the same config and version tag.
> The current downgrade code doesn't "break" after finding its first match:
> {code}
>       for(ClusterConfigMappingEntity configMappingEntity: configMappingEntities){
>         String type = configMappingEntity.getType();
>         String tag =  configMappingEntity.getTag();
>         for (ClusterConfigMappingEntity latest : latestConfigMappingByStack) {
>           String latestType = latest.getType();
>           String latestTag = latest.getTag();
>           // find the latest config of a given mapping entity
>           if (StringUtils.equals(type, latestType) && StringUtils.equals(tag,
latestTag)) {
>             LOG.info("{} with version tag {} is selected for stack {}", type, tag, stackId.toString());
>             configMappingEntity.setSelected(1);
>           }
>         }
>       }
> {code}
> I'm not sure that breaking will work here since we don't know the ordering. We could
order the results and then break on the first match. In any event, it's a problem. But it's
only a problem for the latest version of a config since that's what we're going to try to
make selected.
> Here's a query from a test cluster which shows that if this cluster were to be downgraded,
5 configs would have multiple mappings created. This is because for the latest config, there
are multiple mappings in {{clusterconfigmapping}}.
> {noformat}
> SELECT
>   mapping.type_name,
>   mapping.version_tag,
>   COUNT(*)
> FROM clusterconfigmapping mapping
> JOIN (SELECT
>   config.type_name,
>   config.version_tag,
>   MAX(config.version) AS latest_version
> FROM clusterconfig config
> GROUP BY config.type_name) AS latestConfig
>   ON latestConfig.type_name = mapping.type_name
>   AND latestConfig.version_tag = mapping.version_tag
> GROUP BY
>     type_name, version_tag  
> HAVING COUNT(*) > 1
> +-------------------------------+----------------------+----------+
> | type_name                     | version_tag          | COUNT(*) |
> +-------------------------------+----------------------+----------+
> | hcat-env                      | version1             |        5 |
> | hive-exec-log4j               | version1             |        5 |
> | hive-log4j                    | version1             |        5 |
> | ranger-hive-plugin-properties | version1436918769763 |        3 |
> | webhcat-env                   | version1             |        5 |
> +-------------------------------+----------------------+----------+
> 5 rows in set (0.00 sec)
> {noformat}
> STR:
> # Install a cluster, and instrument the {{clusterconfigmapping}} table to have multiple
mappings for a given type (with only 1 currently selected).
> # Upgrade and then downgrade 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message