Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id A539E200B5B for ; Fri, 5 Aug 2016 12:51:42 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id A3D7D160A8E; Fri, 5 Aug 2016 10:51:42 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 74CC3160A6D for ; Fri, 5 Aug 2016 12:51:41 +0200 (CEST) Received: (qmail 33755 invoked by uid 500); 5 Aug 2016 10:51:40 -0000 Mailing-List: contact commits-help@atlas.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@atlas.incubator.apache.org Delivered-To: mailing list commits@atlas.incubator.apache.org Received: (qmail 33746 invoked by uid 99); 5 Aug 2016 10:51:40 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Aug 2016 10:51:40 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 446A41863AC for ; Fri, 5 Aug 2016 10:51:40 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.646 X-Spam-Level: X-Spam-Status: No, score=-4.646 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-1.426] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Of9k3m-PG0SW for ; Fri, 5 Aug 2016 10:51:35 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id D460C5FCE6 for ; Fri, 5 Aug 2016 10:51:34 +0000 (UTC) Received: (qmail 33366 invoked by uid 99); 5 Aug 2016 10:51:34 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 05 Aug 2016 10:51:34 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 02C6FE08E8; Fri, 5 Aug 2016 10:51:33 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: shwethags@apache.org To: commits@atlas.incubator.apache.org Date: Fri, 05 Aug 2016 10:51:34 -0000 Message-Id: <20c8a01ae7ad4266955154c407ae3757@git.apache.org> In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [2/2] incubator-atlas git commit: ATLAS-1096 Modify HveMetaStoreBridge.import to use getEntity instead of DSL (sumasai via shwethags) archived-at: Fri, 05 Aug 2016 10:51:42 -0000 ATLAS-1096 Modify HveMetaStoreBridge.import to use getEntity instead of DSL (sumasai via shwethags) Project: http://git-wip-us.apache.org/repos/asf/incubator-atlas/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/3f51160f Tree: http://git-wip-us.apache.org/repos/asf/incubator-atlas/tree/3f51160f Diff: http://git-wip-us.apache.org/repos/asf/incubator-atlas/diff/3f51160f Branch: refs/heads/master Commit: 3f51160f4a89536fa95cc167abbae4fab362e127 Parents: 38fd4f3 Author: Shwetha GS Authored: Fri Aug 5 11:48:37 2016 +0530 Committer: Shwetha GS Committed: Fri Aug 5 16:21:19 2016 +0530 ---------------------------------------------------------------------- .../atlas/hive/bridge/HiveMetaStoreBridge.java | 72 +++++++------------- .../hive/bridge/HiveMetaStoreBridgeTest.java | 64 +++++++++-------- .../hive/bridge/HiveMetastoreBridgeIT.java | 6 ++ release-log.txt | 1 + 4 files changed, 69 insertions(+), 74 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/3f51160f/addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java ---------------------------------------------------------------------- diff --git a/addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java b/addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java index 8d24a67..270ecf4 100755 --- a/addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java +++ b/addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java @@ -54,8 +54,6 @@ import org.apache.hadoop.hive.ql.metadata.HiveException; import org.apache.hadoop.hive.ql.metadata.Table; import org.apache.hadoop.hive.ql.session.SessionState; import org.apache.hadoop.security.UserGroupInformation; -import org.codehaus.jettison.json.JSONArray; -import org.codehaus.jettison.json.JSONObject; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -72,7 +70,6 @@ public class HiveMetaStoreBridge { public static final String HIVE_CLUSTER_NAME = "atlas.cluster.name"; public static final String DEFAULT_CLUSTER_NAME = "primary"; public static final String DESCRIPTION_ATTR = "description"; - public static final String SEARCH_ENTRY_GUID_ATTR = "__guid"; public static final String TEMP_TABLE_PREFIX = "_temp-"; @@ -214,30 +211,7 @@ public class HiveMetaStoreBridge { LOG.debug("Getting reference for database {}", databaseName); String typeName = HiveDataTypes.HIVE_DB.getName(); - String dslQuery = getDatabaseDSLQuery(clusterName, databaseName, typeName); - return getEntityReferenceFromDSL(typeName, dslQuery); - } - - static String getDatabaseDSLQuery(String clusterName, String databaseName, String typeName) { - return String.format("%s where %s = '%s' and %s = '%s'", typeName, AtlasClient.NAME, - databaseName.toLowerCase(), AtlasConstants.CLUSTER_NAME_ATTRIBUTE, clusterName); - } - - private Referenceable getEntityReferenceFromDSL(String typeName, String dslQuery) throws Exception { - AtlasClient dgiClient = getAtlasClient(); - JSONArray results = dgiClient.searchByDSL(dslQuery, 1, 0); - if (results.length() == 0) { - return null; - } else { - String guid; - JSONObject row = results.getJSONObject(0); - if (row.has("$id$")) { - guid = row.getJSONObject("$id$").getString("id"); - } else { - guid = row.getJSONObject("_col_0").getString("id"); - } - return new Referenceable(guid, typeName, null); - } + return getEntityReference(typeName, getDBQualifiedName(clusterName, databaseName)); } /** @@ -253,11 +227,16 @@ public class HiveMetaStoreBridge { private String getCreateTableString(Table table, String location){ String colString = ""; List colList = table.getAllCols(); - for(FieldSchema col:colList){ - colString += col.getName() + " " + col.getType() + ","; + if ( colList != null) { + for (FieldSchema col : colList) { + colString += col.getName() + " " + col.getType() + ","; + } + if (colList.size() > 0) { + colString = colString.substring(0, colString.length() - 1); + colString = "(" + colString + ")"; + } } - colString = colString.substring(0, colString.length() - 1); - String query = "create external table " + table.getTableName() + "(" + colString + ")" + + String query = "create external table " + table.getTableName() + colString + " location '" + location + "'"; return query; } @@ -293,7 +272,7 @@ public class HiveMetaStoreBridge { Table table = hiveClient.getTable(databaseName, tableName); Referenceable tableReferenceable = registerTable(databaseReferenceable, table); if (table.getTableType() == TableType.EXTERNAL_TABLE) { - String tableQualifiedName = getTableQualifiedName(clusterName, table); + String tableQualifiedName = getTableProcessQualifiedName(clusterName, table); Referenceable process = getProcessReference(tableQualifiedName); if (process == null) { LOG.info("Attempting to register create table process for {}", tableQualifiedName); @@ -347,25 +326,26 @@ public class HiveMetaStoreBridge { LOG.debug("Getting reference for table {}.{}", hiveTable.getDbName(), hiveTable.getTableName()); String typeName = HiveDataTypes.HIVE_TABLE.getName(); - String dslQuery = getTableDSLQuery(getClusterName(), hiveTable.getDbName(), hiveTable.getTableName(), typeName, hiveTable.isTemporary()); - return getEntityReferenceFromDSL(typeName, dslQuery); + String tblQualifiedName = getTableQualifiedName(getClusterName(), hiveTable.getDbName(), hiveTable.getTableName()); + return getEntityReference(typeName, tblQualifiedName); + } + + private Referenceable getEntityReference(final String typeName, final String tblQualifiedName) throws AtlasServiceException { + AtlasClient dgiClient = getAtlasClient(); + try { + return dgiClient.getEntity(typeName, AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, tblQualifiedName); + } catch (AtlasServiceException e) { + if(e.getStatus() == ClientResponse.Status.NOT_FOUND) { + return null; + } + throw e; + } } private Referenceable getProcessReference(String qualifiedName) throws Exception{ LOG.debug("Getting reference for process {}", qualifiedName); String typeName = HiveDataTypes.HIVE_PROCESS.getName(); - String dslQuery = getProcessDSLQuery(typeName, qualifiedName); - return getEntityReferenceFromDSL(typeName, dslQuery); - } - - static String getProcessDSLQuery(String typeName, String qualifiedName) throws Exception{ - String dslQuery = String.format("%s as t where qualifiedName = '%s'", typeName, qualifiedName); - return dslQuery; - } - - static String getTableDSLQuery(String clusterName, String dbName, String tableName, String typeName, boolean isTemporary) { - String entityName = getTableQualifiedName(clusterName, dbName, tableName, isTemporary); - return String.format("%s as t where qualifiedName = '%s'", typeName, entityName); + return getEntityReference(typeName, qualifiedName); } /** http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/3f51160f/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java ---------------------------------------------------------------------- diff --git a/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java b/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java index f8aa93a..e488f93 100644 --- a/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java +++ b/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridgeTest.java @@ -23,8 +23,10 @@ import org.apache.atlas.AtlasServiceException; import org.apache.atlas.hive.model.HiveDataModelGenerator; import org.apache.atlas.hive.model.HiveDataTypes; import org.apache.atlas.typesystem.Referenceable; +import org.apache.hadoop.fs.Path; import org.apache.hadoop.hive.metastore.TableType; import org.apache.hadoop.hive.metastore.api.Database; +import org.apache.hadoop.hive.metastore.api.FieldSchema; import org.apache.hadoop.hive.ql.metadata.Hive; import org.apache.hadoop.hive.ql.metadata.HiveException; import org.apache.hadoop.hive.ql.metadata.Partition; @@ -95,13 +97,13 @@ public class HiveMetaStoreBridgeTest { returnExistingDatabase(TEST_DB_NAME, atlasClient, CLUSTER_NAME); // return existing table - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getTableDSLQuery(CLUSTER_NAME, TEST_DB_NAME, TEST_TABLE_NAME, - HiveDataTypes.HIVE_TABLE.getName(), false), 1, 0)).thenReturn( - getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); + when(atlasClient.getEntity(HiveDataTypes.HIVE_TABLE.getName(), + AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, TEST_DB_NAME, TEST_TABLE_NAME))) + .thenReturn(getEntityReference(HiveDataTypes.HIVE_TABLE.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77")); when(atlasClient.getEntity("82e06b34-9151-4023-aa9d-b82103a50e77")).thenReturn(createTableReference()); - String processQualifiedName = HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, hiveTables.get(0)); - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getProcessDSLQuery(HiveDataTypes.HIVE_PROCESS.getName(), - processQualifiedName), 1, 0)).thenReturn(getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); + String processQualifiedName = HiveMetaStoreBridge.getTableProcessQualifiedName(CLUSTER_NAME, hiveTables.get(0)); + when(atlasClient.getEntity(HiveDataTypes.HIVE_PROCESS.getName(), + AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, processQualifiedName)).thenReturn(getEntityReference(HiveDataTypes.HIVE_PROCESS.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77")); HiveMetaStoreBridge bridge = new HiveMetaStoreBridge(CLUSTER_NAME, hiveClient, atlasClient); bridge.importHiveMetadata(true); @@ -114,9 +116,10 @@ public class HiveMetaStoreBridgeTest { private void returnExistingDatabase(String databaseName, AtlasClient atlasClient, String clusterName) throws AtlasServiceException, JSONException { - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getDatabaseDSLQuery(clusterName, databaseName, - HiveDataTypes.HIVE_DB.getName()), 1, 0)).thenReturn( - getEntityReference("72e06b34-9151-4023-aa9d-b82103a50e76")); + when(atlasClient.getEntity( + HiveDataTypes.HIVE_DB.getName(), AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, + HiveMetaStoreBridge.getDBQualifiedName(clusterName, databaseName))).thenReturn( + getEntityReference(HiveDataTypes.HIVE_DB.getName(), "72e06b34-9151-4023-aa9d-b82103a50e76")); } private List setupTables(Hive hiveClient, String databaseName, String... tableNames) throws HiveException { @@ -144,12 +147,12 @@ public class HiveMetaStoreBridgeTest { returnExistingDatabase(TEST_DB_NAME, atlasClient, CLUSTER_NAME); - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getTableDSLQuery(CLUSTER_NAME, TEST_DB_NAME, - TEST_TABLE_NAME, HiveDataTypes.HIVE_TABLE.getName(), false), 1, 0)).thenReturn( - getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); - String processQualifiedName = HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, hiveTable); - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getProcessDSLQuery(HiveDataTypes.HIVE_PROCESS.getName(), - processQualifiedName), 1, 0)).thenReturn(getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); + when(atlasClient.getEntity(HiveDataTypes.HIVE_TABLE.getName(), AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, + HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, TEST_DB_NAME, TEST_TABLE_NAME))).thenReturn( + getEntityReference(HiveDataTypes.HIVE_TABLE.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77")); + String processQualifiedName = HiveMetaStoreBridge.getTableProcessQualifiedName(CLUSTER_NAME, hiveTable); + when(atlasClient.getEntity(HiveDataTypes.HIVE_PROCESS.getName(), AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, + processQualifiedName)).thenReturn(getEntityReference(HiveDataTypes.HIVE_PROCESS.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77")); when(atlasClient.getEntity("82e06b34-9151-4023-aa9d-b82103a50e77")).thenReturn(createTableReference()); Partition partition = mock(Partition.class); @@ -176,13 +179,13 @@ public class HiveMetaStoreBridgeTest { returnExistingDatabase(TEST_DB_NAME, atlasClient, CLUSTER_NAME); when(hiveClient.getTable(TEST_DB_NAME, TEST_TABLE_NAME)).thenThrow(new RuntimeException("Timeout while reading data from hive metastore")); - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getTableDSLQuery(CLUSTER_NAME, TEST_DB_NAME, - table2Name, HiveDataTypes.HIVE_TABLE.getName(), false), 1, 0)).thenReturn( - getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); + when(atlasClient.getEntity(HiveDataTypes.HIVE_TABLE.getName(), AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, TEST_DB_NAME, + table2Name))).thenReturn( + getEntityReference(HiveDataTypes.HIVE_TABLE.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77")); when(atlasClient.getEntity("82e06b34-9151-4023-aa9d-b82103a50e77")).thenReturn(createTableReference()); - String processQualifiedName = HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, hiveTables.get(1)); - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getProcessDSLQuery(HiveDataTypes.HIVE_PROCESS.getName(), - processQualifiedName), 1, 0)).thenReturn(getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); + String processQualifiedName = HiveMetaStoreBridge.getTableProcessQualifiedName(CLUSTER_NAME, hiveTables.get(1)); + when(atlasClient.getEntity(HiveDataTypes.HIVE_PROCESS.getName(), AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, + processQualifiedName)).thenReturn(getEntityReference(HiveDataTypes.HIVE_PROCESS.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77")); HiveMetaStoreBridge bridge = new HiveMetaStoreBridge(CLUSTER_NAME, hiveClient, atlasClient); try { @@ -201,13 +204,13 @@ public class HiveMetaStoreBridgeTest { returnExistingDatabase(TEST_DB_NAME, atlasClient, CLUSTER_NAME); when(hiveClient.getTable(TEST_DB_NAME, TEST_TABLE_NAME)).thenThrow(new RuntimeException("Timeout while reading data from hive metastore")); - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getTableDSLQuery(CLUSTER_NAME, TEST_DB_NAME, - table2Name, HiveDataTypes.HIVE_TABLE.getName(), false), 10, 0)).thenReturn( - getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); + when(atlasClient.getEntity(HiveDataTypes.HIVE_TABLE.getName(), AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, TEST_DB_NAME, + table2Name))).thenReturn( + getEntityReference(HiveDataTypes.HIVE_TABLE.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77")); when(atlasClient.getEntity("82e06b34-9151-4023-aa9d-b82103a50e77")).thenReturn(createTableReference()); String processQualifiedName = HiveMetaStoreBridge.getTableQualifiedName(CLUSTER_NAME, hiveTables.get(1)); - when(atlasClient.searchByDSL(HiveMetaStoreBridge.getProcessDSLQuery(HiveDataTypes.HIVE_PROCESS.getName(), - processQualifiedName), 10, 0)).thenReturn(getEntityReference("82e06b34-9151-4023-aa9d-b82103a50e77")); + when(atlasClient.getEntity(HiveDataTypes.HIVE_PROCESS.getName(), AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, + processQualifiedName)).thenReturn(getEntityReference(HiveDataTypes.HIVE_PROCESS.getName(), "82e06b34-9151-4023-aa9d-b82103a50e77"));; HiveMetaStoreBridge bridge = new HiveMetaStoreBridge(CLUSTER_NAME, hiveClient, atlasClient); try { @@ -218,8 +221,8 @@ public class HiveMetaStoreBridgeTest { } } - private JSONArray getEntityReference(String id) throws JSONException { - return new JSONArray(String.format("[{\"$id$\":{\"id\":\"%s\"}}]", id)); + private Referenceable getEntityReference(String typeName, String id) throws JSONException { + return new Referenceable(id, typeName, null); } private Referenceable createTableReference() { @@ -232,7 +235,12 @@ public class HiveMetaStoreBridgeTest { private Table createTestTable(String databaseName, String tableName) throws HiveException { Table table = new Table(databaseName, tableName); table.setInputFormatClass(TextInputFormat.class); + table.setFields(new ArrayList() {{ + add(new FieldSchema("col1", "string", "comment1")); + } + }); table.setTableType(TableType.EXTERNAL_TABLE); + table.setDataLocation(new Path("somehdfspath")); return table; } http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/3f51160f/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java ---------------------------------------------------------------------- diff --git a/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java b/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java index 7e658a7..57f93a8 100644 --- a/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java +++ b/addons/hive-bridge/src/test/java/org/apache/atlas/hive/bridge/HiveMetastoreBridgeIT.java @@ -52,6 +52,8 @@ public class HiveMetastoreBridgeIT extends HiveITBase { assertEquals(outputs.size(), 1); assertEquals(outputs.get(0).getId()._getId(), tableId); + int tableCount = atlasClient.listEntities(HiveDataTypes.HIVE_TABLE.getName()).size(); + //Now import using import tool - should be no-op hiveMetaStoreBridge.importTable(atlasClient.getEntity(dbId), DEFAULT_DB, tableName, true); String tableId2 = assertTableIsRegistered(DEFAULT_DB, tableName); @@ -61,6 +63,10 @@ public class HiveMetastoreBridgeIT extends HiveITBase { AtlasClient.REFERENCEABLE_ATTRIBUTE_NAME, getTableProcessQualifiedName(DEFAULT_DB, tableName), null); assertEquals(processId2, processId); + + //assert that table is de-duped and no new entity is created + int newTableCount = atlasClient.listEntities(HiveDataTypes.HIVE_TABLE.getName()).size(); + assertEquals(newTableCount, tableCount); } @Test http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/3f51160f/release-log.txt ---------------------------------------------------------------------- diff --git a/release-log.txt b/release-log.txt index 57aef52..5e0d4af 100644 --- a/release-log.txt +++ b/release-log.txt @@ -6,6 +6,7 @@ INCOMPATIBLE CHANGES: ALL CHANGES: +ATLAS-1096 Modify HveMetaStoreBridge.import to use getEntity instead of DSL (sumasai via shwethags) ATLAS-1091 Improvement in DSL search functionality. (kevalbhatt) ATLAS-1080 Regression - UI - hive_storagedesc is shown as "undefined" in UI.(kevalbhatt) ATLAS-1089 Storm hook should handle cyclic references in topology object (mneethiraj via sumasai)