atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suma Shivaprasad <sumasai.shivapra...@gmail.com>
Subject Re: Review Request 51939: Framework to apply updates to types in the type-system
Date Tue, 20 Sep 2016 20:28:05 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51939/#review149727
-----------------------------------------------------------




typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java (line 314)
<https://reviews.apache.org/r/51939/#comment217407>

    We would need to audit types as well to 
    know what was the original type definition that was updated and the patch that was applied.


- Suma Shivaprasad


On Sept. 19, 2016, 6:10 p.m., Sarath Kumar Subramanian wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/51939/
> -----------------------------------------------------------
> 
> (Updated Sept. 19, 2016, 6:10 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Shwetha GS, and Suma Shivaprasad.
> 
> 
> Bugs: ATLAS-1174
>     https://issues.apache.org/jira/browse/ATLAS-1174
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> 1. Introduce "version" attribute to all types in the type-system, this helps to track
changes made to the default types (hive, sqoop, falcon and storm types) and user created types.
If version is not mentioned during creation of a type, default version "1.0" is assigned (optional
attribute).
> 2. Using the version attributed for types, introduce a patch framework for type system.
This framework applies patches to a type using its version number and can be used during upgrade
- add new attributes to an existing types and it will be run during atlas startup.
> The sequence of steps:
> a. During atlas startup, check $ATLAS_HOME/models/patches directory for any available
patch files (json files). If there any patch files handle them.
> b. Sample patch json file looks like:
> {
> "patches": [
> { 
> "action": "ADD_ATTRIBUTE",
> "typeName": "hive_column",
> "applyToVersion": "1.0",
> "updateToVersion": "2.0",
> "actionParams": [
> { "name": "position", "dataTypeName": "int", "multiplicity": "optional", "isComposite":
false, "isUnique": false, "isIndexable": false, "reverseAttributeName": null }
> ]
> } ]
> }
> c. The framework updates the type in "typeName" for the matching version number and after
applying the patch, update the version to the one mentioned in "updateToVersion"
> d. The json file can have more than one action (array of actions).
> e. There can be multiple patch json files in the directory and are applied in the sort
order of the filename. eg:
> 001-hive_column_add_position.json
> 002-hive_column_add_anotherattribute.json
> 
> 
> Diffs
> -----
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java d7f9c89 
>   repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java
a94d157 
>   repository/src/main/java/org/apache/atlas/services/AddAttributePatch.java PRE-CREATION

>   repository/src/main/java/org/apache/atlas/services/AtlasTypePatch.java PRE-CREATION

>   repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java 6a937f4

>   typesystem/src/main/java/org/apache/atlas/typesystem/types/AbstractDataType.java fad091d

>   typesystem/src/main/java/org/apache/atlas/typesystem/types/ClassType.java c56987a 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/EnumType.java bdd0a13 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/EnumTypeDefinition.java
6340615 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/HierarchicalType.java 7224699

>   typesystem/src/main/java/org/apache/atlas/typesystem/types/HierarchicalTypeDefinition.java
9a299f0 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/IDataType.java 85ddee7 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/StructType.java 6f40c1d

>   typesystem/src/main/java/org/apache/atlas/typesystem/types/StructTypeDefinition.java
f1ce1b7 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/TraitType.java f23bf5b 
>   typesystem/src/main/java/org/apache/atlas/typesystem/types/TypeSystem.java 70ba89b

>   typesystem/src/main/java/org/apache/atlas/typesystem/types/utils/TypesUtil.java ef8448d

>   typesystem/src/main/scala/org/apache/atlas/typesystem/json/TypesSerialization.scala
5618938 
> 
> Diff: https://reviews.apache.org/r/51939/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Sarath Kumar Subramanian
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message