atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 62244: ATLAS2058 Add description to attributedefs and relationship enddefs
Date Wed, 13 Sep 2017 14:50:11 GMT


> On Sept. 12, 2017, 6:15 p.m., Sarath Subramanian wrote:
> > intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java
> > Lines 275 (patched)
> > <https://reviews.apache.org/r/62244/diff/1/?file=1820214#file1820214line275>
> >
> >     can we include descriptions for existing attributes and relationshipEndDefs
for out of the box models as part of this JIRA?
> 
> David Radley wrote:
>     I think it is better to separate the putting values into the models into separate
Jiras. We can then use those Jiras to review the description content rather than the mechanism
to introduce description.
> 
> Madhan Neethiraj wrote:
>     I agree. However, I would suggest adding 'Map<String, String> aliases' field
to struct-def and attribute-def - to enable alternate names for these to be stored along with
entity-def, classification-def and attribute-def. This should be a trivial update and can
be taken up along with addition of 'description' field.
> 
> David Radley wrote:
>     I am unsure what the keys and values might be for the aliases map you are proposing.
Are you thinking that the aliases would be instance aliases or type aliases? 
>     
>     We are thinking of adding external type identifications (ExternalTypeIdDefs) as a
new array associated with attributes, structures (and the otehr defs). We are thinking the
elements would be specified in the typeDefs in the form:
>     
>     ExternalTypeIdDefs [
>     {   "scheme":"O-DEF",
>         "value":"123-456",
>         "description":"Relational Database",
>         "ResponsibleOrg":"Open Group",
>         "ReferenceDocumentationURL":"http://xxx.yyy.zzz"
>     },
>     {
>     ...
>     }
>     ]
>     
>     Scheme and value would be manditory.
>     
>     One way to handle type aliases would be using the above mechanism: 
>     {   "scheme":"alias",
>         "value":"RDB",
>         "description":"short hand relational database key for consumer A",
>         "ResponsibleOrg":"Consumer A organisation",
>         "ReferenceDocumentationURL":"http://aaa.bbb.bbb"
>     },
>     
>     or
>     
>     {   "scheme":"alias",
>         "value":"RDB"
>     },
>     
>     
>     This would allow these external type identifier metadata values to be visible in
the json models.
>     
>     I was thinking of doing this piece separately in another Jira. The addition of the
code as-is in the Jira will in enable me to add desciptions to the attributes / relationship
ends in the new models. For the existing Hadoop models - it might be worth having someone
more expert in Hadoop filling in those descriptions.
> 
> David Radley wrote:
>     One other thought - if the aliases map are for instances - I wonder if they could
be indexed. I assume they could be specified in a search query.
> 
> Madhan Neethiraj wrote:
>     We discussed about an entity-type or an attribute in Atlas being referred with a
different name in other meta-data standards. The 'alias' field I suggested above is to capture
these names along with entity-def/attribute-def in Atlas. I assume it will be enough to capure
only the name used in other standards; if more richer information is available, how likely
it is to be able to capture all the details available in various standards? To keep it simple,
we can start with only capturing the names used in other meta-data standards.
> 
> David Radley wrote:
>     Hi Madhan,
>     Thanks for your quick reply. We did start with name and value pair, but felt the
additional fields could give real value, for those that use them.  
>     
>     To answer " how likely it is to be able to capture all the details available in various
standards". I think that for the O-DEF content we are thinking about adding to the models,
we could fill out all this content. If other standards do not want to or have this information,
then they can omit the optional fields. The advantage of the "ResponsibleOrg" and "ReferenceDocumentationURL"
is that they leave no doubt as to the author of these names, which is great for auditing and
maybe even lineage visualizations. Just calling this alias may imply this is a display name
- but for the use case I was thinking of, it is actually an external identifier in another
metadata standard.  
>     
>     I hope this approach keeps it simple for people wanting to add name value pairs,
also allowing us to have the values in the model files (a very easy proof point to show we
can work with O-DEF) and also gives us the ability to add the extra provenance information.
Lets discuss further on another Jira if we need to.
> 
> David Radley wrote:
>     Raised Jira https://issues.apache.org/jira/browse/ATLAS-2133 to track the alias /
ExternalTypeIdDefs work

ok. Let's discuss the 'external-id' details via ATLAS-2133.


- Madhan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62244/#review185223
-----------------------------------------------------------


On Sept. 12, 2017, 5:13 p.m., David Radley wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62244/
> -----------------------------------------------------------
> 
> (Updated Sept. 12, 2017, 5:13 p.m.)
> 
> 
> Review request for atlas and Madhan Neethiraj.
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> ATLAS2058 Add description to attributedefs and relationship enddefs
> 
> 
> Diffs
> -----
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasRelationshipEndDef.java 01e5ce738e8a5310cee928560364d4a38f5c6e18

>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 3a5c43ac7e6d0a7f683c24d13d6c841de4d66a14

>   repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
c3c42be86d8c9fd282fdc1428c8fd78e338a3cfb 
> 
> 
> Diff: https://reviews.apache.org/r/62244/diff/1/
> 
> 
> Testing
> -------
> 
> Create an entity attribute, relationship attribute and a relationship end with a description
and then issue getTypeDefs to check they are there.
> 
> 
> Thanks,
> 
> David Radley
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message