atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benjamin BONNET (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-1990) Regression :Traits Attribute that References Another Entity
Date Tue, 25 Jul 2017 12:09:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-1990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16099937#comment-16099937
] 

Benjamin BONNET commented on ATLAS-1990:
----------------------------------------

Hi David,
thank you for your answer.
This is an example of use case I am trying to implement : I have tables A and B that have
a one to many relationship based on a primary/foreign key. So I would like to tag the foreign
key column (hive_column in B) with a trait that holds a reference to the correspunding primary
key (hive_column in A).
I have already made this case work with Atlas 0.5 but upgrading to 0.8 breaks it.
Regards.

> Regression :Traits Attribute that References Another Entity
> -----------------------------------------------------------
>
>                 Key: ATLAS-1990
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1990
>             Project: Atlas
>          Issue Type: Bug
>          Components:  atlas-core
>    Affects Versions: 0.8-incubating
>         Environment: HDP2.6
>            Reporter: Benjamin BONNET
>
> Hi,
> I defined a trait that includes one attribute. That attribute has type "hive_column",
i.e. it references a hive_column entity.
> Using Altas 0.5 on a HDP 2.4, I could attach that trait to an entity so that it refers
a column. But using Atlas 0.8 on a HDP 2.6 that does not work anymore (I get a NullPointerException).
> I tried both V1 and V2 REST API and got the same error.
> Here is a full sample to reproduce that issue.
> Trait definition : 
> {noformat}
> {"enumTypes":[],
>  "structTypes":[],
>  "traitTypes":[{
>    "superTypes":[],
>    "hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType",
>    "typeName":"Reference",
>    "typeDescription":"reference test",
>    "attributeDefinitions":[{"name":"ref",
>                            "dataTypeName":"hive_column",
>                           "multiplicity":"required",
>                           "isComposite":false,
>                           "isUnique":false,
>                           "isIndexable":true,
>                           "reverseAttributeName":null
>                          }
> ]}],
>   "classTypes":[]}
> {noformat}
> Then I use curl to create that type (ref file contains the traits definition):
> {noformat}
> curl -v -u admin:admin -X POST -d "`cat ref`" -H 'Content-Type: application/json' 'http://127.0.0.1:21000/api/atlas/types'
> {noformat}
> Using that command, trait type is created OK on both 0.5 and 0.8.
> Now, I can tag an entity on Atlas 0.5 :
> {noformat}
> curl -v -u admin:admin -X POST -d '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
"typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"9e2b0059-8e75-42ce-a32c-fe9a6157caed","version":0,"typeName":"hive_column"}}}'
-H "Content-Type: application/json" "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits"
> {noformat}
> I get a HTTP201 :
> < HTTP/1.1 201 Created
> < Date: Tue, 25 Jul 2017 09:11:23 GMT
> < Location: http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits/23d1c3e7-7ad9-47b6-ac54-1d507119d00d
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-15 - 6d4b21dd-af88-46f3-91f4-5b3b20b6840c","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d"}
> And if I check my entity traits, I can see that tag.
> {noformat}
> curl -v -u admin:admin -X GET  "http://127.0.0.1:21000/api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits"
                    * About to connect() to 127.0.0.1 port 21000 (#0)
> *   Trying 127.0.0.1... connected
> * Connected to 127.0.0.1 (127.0.0.1) port 21000 (#0)
> * Server auth using Basic with user 'admin'
> > GET /api/atlas/entities/23d1c3e7-7ad9-47b6-ac54-1d507119d00d/traits HTTP/1.1
> > Authorization: Basic YWRtaW46YWRtaW4=
> > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.19.1 Basic
ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> > Host: 127.0.0.1:21000
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < Date: Tue, 25 Jul 2017 09:13:27 GMT
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"requestId":"qtp999445351-13 - 15a0ce26-484a-4181-9f5a-742227be5d09","GUID":"23d1c3e7-7ad9-47b6-ac54-1d507119d00d","results":["tag1","tag2","Reference","tag3"],"count":4}
> {noformat}
> So far, so good, but If I try to do the same thing Atlas 0.8, i.e. tagging an entity
with that trait :
> {noformat}
> curl -v -u admin:admin -X POST -d '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
"typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"d5a507e6-0752-4922-a524-cac6e8606f7d","version":0,"typeName":"hive_column"}}}'
-H "Content-Type: application/json" "http://127.0.0.1:21000/api/atlas/entities/f79fa725-f628-4672-8b7e-1568ebdf764a/traits"
> {noformat}
> I get a HTTP500
> {noformat}
> < HTTP/1.1 500 Internal Server Error
> < Date: Mon, 24 Jul 2017 17:28:15 GMT
> < Set-Cookie: ATLASSESSIONID=hnk1soc7nw8v7e4me880rct0;Path=/;HttpOnly
> < Expires: Thu, 01 Jan 1970 00:00:00 GMT
> < X-Frame-Options: DENY
> < Content-Type: application/json; charset=UTF-8
> < Transfer-Encoding: chunked
> < Server: Jetty(9.2.12.v20150709)
> <
> * Connection #0 to host 127.0.0.1 left intact
> * Closing connection #0
> {"error":"Failed with java.lang.NullPointerException"}
> {noformat}
> And I get more details on the error in Atlas log file :
> {noformat}
> 2017-07-24 17:28:17,070 DEBUG - [pool-2-thread-4 - 28462fd0-634f-454d-85f4-181f63bc626d:]
~ ==> mapObjectIdValue(org.apache.atlas.repository.store.graph.v1.AttributeMutationContext@1008a43e)
(EntityGraphMapper:437)
> 2017-07-24 17:28:17,071 ERROR - [pool-2-thread-4 - 28462fd0-634f-454d-85f4-181f63bc626d:]
~ graph rollback due to exception  (GraphTransactionInterceptor:62)
> java.lang.NullPointerException
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapObjectIdValue(EntityGraphMapper.java:444)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:301)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:260)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:221)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapClassification(EntityGraphMapper.java:925)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.addClassifications(EntityGraphMapper.java:916)
>         at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.addClassification(AtlasEntityStoreV1.java:470)
>         at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:51)
>         at org.apache.atlas.web.resources.EntityResource.addTrait(EntityResource.java:1045)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ......
> 2017-07-24 17:28:17,072 ERROR - [pool-2-thread-4 - 28462fd0-634f-454d-85f4-181f63bc626d:]
~ Unable to add trait for entity=f79fa725-f628-4672-8b7e-1568ebdf764a traitDef={"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
"typeName":"Reference","values":{"ref":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"d5a507e6-0752-4922-a524-cac6e8606f7d","version":0,"typeName":"hive_column"}}}
(EntityResource:1065)
> java.lang.NullPointerException
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapObjectIdValue(EntityGraphMapper.java:444)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:301)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:260)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:221)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapClassification(EntityGraphMapper.java:925)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.addClassifications(EntityGraphMapper.java:916)
>         at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.addClassification(AtlasEntityStoreV1.java:470)
>         at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:51)
>         at org.apache.atlas.web.resources.EntityResource.addTrait(EntityResource.java:1045)
> ........
> {noformat}
> I suspected it was because V1 API became kind of obsolete, so I tried to attach the tag
using V2 API :
> {noformat}
> curl -v -u admin:admin -X POST -d '[{"typeName":"Relational", "attributes":{"related":[{"guid":"d5a507e6-0752-4922-a524-cac6e8606f7d","typeName":"hive_table"}]}}]'
-H 'Content-Type: application/json' 'http://127.0.0.1:21000/api/atlas/v2/entity/guid/f79fa725-f628-4672-8b7e-1568ebdf764a/classifications'
> {noformat}
> but I get the same HTTP500 error, and the log file shows a stack trace nearly identical.
> {noformat}
> 2017-07-24 17:35:01,679 ERROR - [pool-2-thread-4 - 3d34de5d-1979-4f8a-8fba-3d85c9495bf7:]
~ Error handling a request: b47a4619343bf332
> (ExceptionMapperUtil:32)
> java.lang.NullPointerException
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapObjectIdValue(EntityGraphMapper.java:444)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapCollectionElementsToVertex(EntityGraphMapper.java:644)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapArrayValue(EntityGraphMapper.java:568)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapToVertexByTypeCategory(EntityGraphMapper.java:318)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttribute(EntityGraphMapper.java:260)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapAttributes(EntityGraphMapper.java:221)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.mapClassification(EntityGraphMapper.java:925)
>         at org.apache.atlas.repository.store.graph.v1.EntityGraphMapper.addClassifications(EntityGraphMapper.java:916)
>         at org.apache.atlas.repository.store.graph.v1.AtlasEntityStoreV1.addClassifications(AtlasEntityStoreV1.java:440)
>         at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:51)
>         at org.apache.atlas.web.rest.EntityREST.addClassifications(EntityREST.java:349)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message