Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 62137200BF4 for ; Fri, 23 Dec 2016 02:46:35 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 607C9160B27; Fri, 23 Dec 2016 01:46:35 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 6677A160B3E for ; Fri, 23 Dec 2016 02:46:34 +0100 (CET) Received: (qmail 48860 invoked by uid 500); 23 Dec 2016 01:46:33 -0000 Mailing-List: contact commits-help@atlas.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@atlas.incubator.apache.org Delivered-To: mailing list commits@atlas.incubator.apache.org Received: (qmail 48641 invoked by uid 99); 23 Dec 2016 01:46:33 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Dec 2016 01:46:33 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id C71CBC10AC for ; Fri, 23 Dec 2016 01:46:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -6.219 X-Spam-Level: X-Spam-Status: No, score=-6.219 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id 3rUpENLhpZSM for ; Fri, 23 Dec 2016 01:46:31 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with SMTP id 13DB55F5F8 for ; Fri, 23 Dec 2016 01:46:30 +0000 (UTC) Received: (qmail 48541 invoked by uid 99); 23 Dec 2016 01:46:30 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Dec 2016 01:46:30 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 7ABFDDFCFD; Fri, 23 Dec 2016 01:46:30 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: madhan@apache.org To: commits@atlas.incubator.apache.org Date: Fri, 23 Dec 2016 01:46:49 -0000 Message-Id: <867be01dc6f9445bab412d0dc6ac309f@git.apache.org> In-Reply-To: References: X-Mailer: ASF-Git Admin Mailer Subject: [20/50] [abbrv] incubator-atlas git commit: ATLAS-1122 Change trait edge labels to have trait name alone (sumasai) archived-at: Fri, 23 Dec 2016 01:46:35 -0000 ATLAS-1122 Change trait edge labels to have trait name alone (sumasai) (cherry picked from commit eb98bfe78942caf1fa630e35f619f3a4f2f61856) Project: http://git-wip-us.apache.org/repos/asf/incubator-atlas/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/dea202ca Tree: http://git-wip-us.apache.org/repos/asf/incubator-atlas/tree/dea202ca Diff: http://git-wip-us.apache.org/repos/asf/incubator-atlas/diff/dea202ca Branch: refs/heads/0.7-incubating Commit: dea202caffc68237e78a01d96cd24e71950c76d0 Parents: cf433a5 Author: Suma Shivaprasad Authored: Wed Aug 17 09:29:26 2016 -0700 Committer: Madhan Neethiraj Committed: Thu Dec 22 15:21:22 2016 -0800 ---------------------------------------------------------------------- release-log.txt | 1 + .../atlas/repository/graph/GraphHelper.java | 2 +- .../query/GraphPersistenceStrategies.scala | 2 +- .../GraphBackedMetadataRepositoryTest.java | 56 +++++++++++--------- .../apache/atlas/query/HiveTitanSample.scala | 2 +- 5 files changed, 34 insertions(+), 29 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/dea202ca/release-log.txt ---------------------------------------------------------------------- diff --git a/release-log.txt b/release-log.txt index 7ffb5dc..b6c6a55 100644 --- a/release-log.txt +++ b/release-log.txt @@ -28,6 +28,7 @@ ATLAS-409 Atlas will not import avro tables with schema read from a file (dosset ATLAS-379 Create sqoop and falcon metadata addons (venkatnrangan,bvellanki,sowmyaramesh via shwethags) ALL CHANGES: +ATLAS-1122 Change trait edge labels to have trait name alone (sumasai) ATLAS-1121 NPE while submitting topology in StormHook (ayubkhan via sumasai) ATLAS-1111 Data loss is observed when atlas is restarted while hive_table metadata ingestion into kafka topic is in-progress(shwethags via sumasai) ATLAS-1108: updated references to atlas.rest.address to handle multiple URLs http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/dea202ca/repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java ---------------------------------------------------------------------- diff --git a/repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java b/repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java index 6efb22a..2ef50c9 100755 --- a/repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java +++ b/repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java @@ -374,7 +374,7 @@ public final class GraphHelper { } public static String getTraitLabel(String typeName, String attrName) { - return typeName + "." + attrName; + return attrName; } public static List getTraitNames(Vertex entityVertex) { http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/dea202ca/repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala ---------------------------------------------------------------------- diff --git a/repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala b/repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala index 21c152c..f774d97 100755 --- a/repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala +++ b/repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala @@ -203,7 +203,7 @@ object GraphPersistenceStrategy1 extends GraphPersistenceStrategies { val fieldPrefixInSelect = "it" - def traitLabel(cls: IDataType[_], traitName: String) = s"${cls.getName}.$traitName" + def traitLabel(cls: IDataType[_], traitName: String) = s"$traitName" def fieldNameInVertex(dataType: IDataType[_], aInfo: AttributeInfo) = GraphHelper.getQualifiedFieldName(dataType, aInfo.name) http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/dea202ca/repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java ---------------------------------------------------------------------- diff --git a/repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java b/repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java index 1ebf3ae..3ca1f78 100755 --- a/repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java +++ b/repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java @@ -43,6 +43,7 @@ import org.apache.atlas.typesystem.Struct; import org.apache.atlas.typesystem.exception.EntityNotFoundException; import org.apache.atlas.typesystem.exception.TraitNotFoundException; import org.apache.atlas.typesystem.persistence.Id; +import org.apache.atlas.typesystem.types.AttributeDefinition; import org.apache.atlas.typesystem.types.ClassType; import org.apache.atlas.typesystem.types.DataTypes; import org.apache.atlas.typesystem.types.HierarchicalTypeDefinition; @@ -77,6 +78,7 @@ import static org.apache.atlas.typesystem.types.utils.TypesUtil.createClassTypeD import static org.apache.atlas.typesystem.types.utils.TypesUtil.createUniqueRequiredAttrDef; import static org.testng.Assert.assertEquals; import static org.testng.Assert.assertNotEquals; +import static org.testng.Assert.assertNotNull; import static org.testng.Assert.assertTrue; /** @@ -136,31 +138,35 @@ public class GraphBackedMetadataRepositoryTest { @Test //In some cases of parallel APIs, the edge is added, but get edge by label doesn't return the edge. ATLAS-1104 public void testConcurrentCalls() throws Exception { - Referenceable dbInstance = new Referenceable(TestUtils.DATABASE_TYPE); - dbInstance.set("name", randomString()); - dbInstance.set("description", "foo database"); - final String id1 = createEntity(dbInstance).get(0); + final HierarchicalTypeDefinition refType = + createClassTypeDef(randomString(), ImmutableSet.of()); + HierarchicalTypeDefinition type = + createClassTypeDef(randomString(), ImmutableSet.of(), + new AttributeDefinition("ref", refType.typeName, Multiplicity.OPTIONAL, true, null)); + typeSystem.defineClassType(refType); + typeSystem.defineClassType(type); + + String refId1 = createEntity(new Referenceable(refType.typeName)).get(0); + String refId2 = createEntity(new Referenceable(refType.typeName)).get(0); - dbInstance.set("name", randomString()); - final String id2 = createEntity(dbInstance).get(0); + final Referenceable instance1 = new Referenceable(type.typeName); + instance1.set("ref", new Referenceable(refId1, refType.typeName, null)); - TraitType piiType = typeSystem.getDataType(TraitType.class, TestUtils.PII); - final ITypedStruct trait = piiType.convert(new Struct(TestUtils.PII), Multiplicity.REQUIRED); + final Referenceable instance2 = new Referenceable(type.typeName); + instance2.set("ref", new Referenceable(refId2, refType.typeName, null)); ExecutorService executor = Executors.newFixedThreadPool(3); List> futures = new ArrayList<>(); futures.add(executor.submit(new Callable() { @Override public Object call() throws Exception { - repositoryService.addTrait(id1, trait); - return null; + return createEntity(instance1).get(0); } })); futures.add(executor.submit(new Callable() { @Override public Object call() throws Exception { - repositoryService.addTrait(id2, trait); - return null; + return createEntity(instance2).get(0); } })); futures.add(executor.submit(new Callable() { @@ -170,25 +176,23 @@ public class GraphBackedMetadataRepositoryTest { } })); - for (Future future : futures) { - future.get(); - } + String id1 = (String) futures.get(0).get(); + String id2 = (String) futures.get(1).get(); + futures.get(2).get(); executor.shutdown(); - boolean validated1 = assertEdge(id1); - boolean validated2 = assertEdge(id2); - assertNotEquals(validated1, validated2); + boolean validated1 = assertEdge(id1, type.typeName); + boolean validated2 = assertEdge(id2, type.typeName); + assertTrue(validated1 | validated2); } - private boolean assertEdge(String id) throws Exception { + private boolean assertEdge(String id, String typeName) throws Exception { TitanGraph graph = graphProvider.get(); Vertex vertex = graph.query().has(Constants.GUID_PROPERTY_KEY, id).vertices().iterator().next(); - Iterable edges = - vertex.getEdges(Direction.OUT, TestUtils.DATABASE_TYPE + "." + TestUtils.PII); - if(!edges.iterator().hasNext()) { - repositoryService.deleteTrait(id, TestUtils.PII); - List traits = repositoryService.getTraitNames(id); - assertTrue(traits.isEmpty()); + Iterable edges = vertex.getEdges(Direction.OUT, Constants.INTERNAL_PROPERTY_KEY_PREFIX + typeName + ".ref"); + if (!edges.iterator().hasNext()) { + ITypedReferenceableInstance entity = repositoryService.getEntityDefinition(id); + assertNotNull(entity.get("ref")); return true; } return false; @@ -237,7 +241,7 @@ public class GraphBackedMetadataRepositoryTest { public void testGetTraitLabel() throws Exception { Assert.assertEquals( repositoryService.getTraitLabel(typeSystem.getDataType(ClassType.class, TestUtils.TABLE_TYPE), - TestUtils.CLASSIFICATION), TestUtils.TABLE_TYPE + "." + TestUtils.CLASSIFICATION); + TestUtils.CLASSIFICATION), TestUtils.CLASSIFICATION); } @Test http://git-wip-us.apache.org/repos/asf/incubator-atlas/blob/dea202ca/repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala ---------------------------------------------------------------------- diff --git a/repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala b/repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala index 67ce12b..2dfb67a 100755 --- a/repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala +++ b/repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala @@ -125,7 +125,7 @@ object HiveTitanSample { if (traits.isDefined) { traits.get foreach { t => t.toGSon(vertices, edges) - addEdge(t, s"${this.getClass.getSimpleName}.${t.getClass.getSimpleName}", edges) + addEdge(t, s"${t.getClass.getSimpleName}", edges) } } }