atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Ding (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-1955) Validation for Attributes
Date Mon, 21 Aug 2017 23:34:00 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-1955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16136014#comment-16136014
] 

Richard Ding commented on ATLAS-1955:
-------------------------------------

Thanks [~davidrad] and [~ivarea] for your comments and suggestions.

It seems that what we want here is a new custom attribute type defined as a top-level type
in Atlas type system. For example, we can have email and credit card attribute types:
{code}
"attributeTypeDefs": [
  
    {
    
        "name": "email",
    
        "typeVersion": "1.0",
        "baseType": "string"
,   
        "validationType": "regex",
    
        "validator": "[0-9a-z]@[0-9a-z].[0-9a-z]+"
  
    },
  
    {
    
        "name": "credit_card",
    
        "typeVersion": "1.0",
        "baseType": "string",
    
    "validationType": "class",
    
        "validator": "org.apache.atlas.model.validataion.CreditCardValidator"
  
    }

]
{code}
And these custom attribute types then can be used in entity attribute definitions:
{code}
"entityDefs": [
  {
       "name":"Person",
       "superTypes": [
         "Referenceable"
       ],
       "typeVersion":"1.0",
       "attributeDefs":[
         {
           "name":"emailAddresses",
           "typeName":"email",
           "cardinality":"SET",
           "isIndexable":true,
           "isOptional":false,
           "isUnique":false
         },
         {
           "name":"creditCardNumbers",
           "typeName":"credit_card",
           "cardinality":"SET",
           "isIndexable":true,
           "isOptional":true,
           "isUnique":false
         },
	 ……
    }
]
{code}
Here  _attributeTypeDefs_ is used to avoid confusion from _attributeDefs_ defined inside _entityDefs_.

> Validation for Attributes
> -------------------------
>
>                 Key: ATLAS-1955
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1955
>             Project: Atlas
>          Issue Type: New Feature
>          Components:  atlas-core
>    Affects Versions: 0.9-incubating
>            Reporter: Israel Varea
>            Assignee: Richard Ding
>             Fix For: 0.9-incubating
>
>
> It would be very nice that Atlas model could contain a way to represent attribute validation.

> A simple example is that we would like to model a Person, with attributes Name, Email
and Country. Now we would like to specify that Email has to follow a specific regular expression,
so it would be nice if we could set Email -> hasValidation -> EmailRegex, with EmailRegex
having:
> Name: Email Regular Expresion
> Expression: /[0-9a-z]+@[0-9a-z]+.[0-9a-z]+/
> For more complex types of validation, e.g. checking card number validity, it could be
added some external validator function/service.
> Name: Credit Card Number Validator
> Validator: org.apache.atlas.validators.creditcard or https://host:port/creditCardValidator
> For validations from a reference table, for example a country name, it could be:
> Name: Country Name Ref Validator
> Reference Column: <country_name_column>
> where <country_name_column> would be an instance of type Hive_Column or HBase_Column.
> Since this is a kind of Standarization, it could be placed in [Area 5|https://cwiki.apache.org/confluence/display/ATLAS/Area+5+-+Standards].
> A similar approach is followed in software [Kylo|https://github.com/Teradata/kylo/tree/master/integrations/spark/spark-validate-cleanse]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message