atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ayub Khan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ATLAS-753) UI: showing "viewOriginalText/viewExpandedText" field which is not part of hive table metadata message and the value will be always be null
Date Fri, 10 Jun 2016 07:09:21 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15324005#comment-15324005
] 

Ayub Khan commented on ATLAS-753:
---------------------------------

This is due to the generic hive data model which includes fields which are needed only for
certain tables. Ok, this is no issue with UI. thanks Hemanth.

> UI: showing "viewOriginalText/viewExpandedText" field which is not part of hive table
metadata message and the value will be always be null
> -------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: ATLAS-753
>                 URL: https://issues.apache.org/jira/browse/ATLAS-753
>             Project: Atlas
>          Issue Type: Bug
>    Affects Versions: 0.7-incubating
>            Reporter: Ayub Khan
>            Assignee: Keval Bhatt
>            Priority: Minor
>             Fix For: 0.7-incubating
>
>
> UI showing "viewOriginalText" field for hive_table which is not part of hive metadata.
> {noformat}
> 0: jdbc:hive2://os-r6-apathan-atlas-erie-nose> describe formatted test_table;
> +-------------------------------+---------------------------------------------------------------------------------------------+-----------------------+--+
> |           col_name            |                                          data_type
                                         |        comment        |
> +-------------------------------+---------------------------------------------------------------------------------------------+-----------------------+--+
> | # col_name                    | data_type                                         
                                         | comment               |
> |                               | NULL                                              
                                         | NULL                  |
> | abc                           | string                                            
                                         |                       |
> |                               | NULL                                              
                                         | NULL                  |
> | # Partition Information       | NULL                                              
                                         | NULL                  |
> | # col_name                    | data_type                                         
                                         | comment               |
> |                               | NULL                                              
                                         | NULL                  |
> | dt                            | date                                              
                                         |                       |
> |                               | NULL                                              
                                         | NULL                  |
> | # Detailed Table Information  | NULL                                              
                                         | NULL                  |
> | Database:                     | default                                           
                                         | NULL                  |
> | Owner:                        | hiveuser                                          
                                         | NULL                  |
> | CreateTime:                   | Fri May 06 05:44:25 UTC 2016                      
                                         | NULL                  |
> | LastAccessTime:               | UNKNOWN                                           
                                         | NULL                  |
> | Protect Mode:                 | None                                              
                                         | NULL                  |
> | Retention:                    | 0                                                 
                                         | NULL                  |
> | Location:                     | hdfs://os-r6-apathan-atlas-erie-nosec-2.openstacklocal:8020/apps/hive/warehouse/test_table
 | NULL                  |
> | Table Type:                   | MANAGED_TABLE                                     
                                         | NULL                  |
> | Table Parameters:             | NULL                                              
                                         | NULL                  |
> |                               | transient_lastDdlTime                             
                                         | 1462513465            |
> |                               | NULL                                              
                                         | NULL                  |
> | # Storage Information         | NULL                                              
                                         | NULL                  |
> | SerDe Library:                | org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
                                         | NULL                  |
> | InputFormat:                  | org.apache.hadoop.mapred.TextInputFormat          
                                         | NULL                  |
> | OutputFormat:                 | org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
                                 | NULL                  |
> | Compressed:                   | No                                                
                                         | NULL                  |
> | Num Buckets:                  | -1                                                
                                         | NULL                  |
> | Bucket Columns:               | []                                                
                                         | NULL                  |
> | Sort Columns:                 | []                                                
                                         | NULL                  |
> | Storage Desc Params:          | NULL                                              
                                         | NULL                  |
> |                               | serialization.format                              
                                         | 1                     |
> +-------------------------------+---------------------------------------------------------------------------------------------+-----------------------+--+
> 31 rows selected (0.377 seconds)
> {noformat}
> Entity create message does not have this data
> {noformat}
> {"entities":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313460476732","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","description":"Default
Hive database","ownerType":2,"qualifiedName":"default@cl1","locationUri":"hdfs://os-r6-apathan-atlas-erie-nosec-2.openstacklocal:8020/apps/hive/warehouse","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{}},{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313487984202","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{"tableType":"MANAGED_TABLE","name":"default.test_table1@cl1","createTime":"2016-05-06T05:52:46.000Z","temporary":false,"db":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313460476732","version":0,"typeName":"hive_db","state":"ACTIVE"},"typeName":"hive_db","values":{"name":"default","description":"Default
Hive database","ownerType":2,"qualifiedName":"default@cl1","locationUri":"hdfs://os-r6-apathan-atlas-erie-nosec-2.openstacklocal:8020/apps/hive/warehouse","owner":"public","clusterName":"cl1","parameters":{}},"traitNames":[],"traits":{}},"retention":0,"tableName":"test_table1","columns":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313488210105","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"abc","qualifiedName":"default.test_table1.abc@cl1","type":"string","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313487984202","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{}}],"lastAccessTime":"2016-05-06T05:52:46.000Z","owner":"hiveuser","sd":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313488120852","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{"location":"hdfs://os-r6-apathan-atlas-erie-nosec-2.openstacklocal:8020/apps/hive/warehouse/test_table1","serdeInfo":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"hive_serde","values":{"serializationLib":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe","parameters":{"serialization.format":"1"}}},"qualifiedName":"default.test_table1@cl1_storage","outputFormat":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","compressed":false,"numBuckets":-1,"inputFormat":"org.apache.hadoop.mapred.TextInputFormat","parameters":{},"storedAsSubDirectories":false,"table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313487984202","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{}},"parameters":{"transient_lastDdlTime":"1462513966"},"partitionKeys":[{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313488152987","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{"name":"dt","qualifiedName":"default.test_table1.dt@cl1","type":"date","table":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-70313487984202","version":0,"typeName":"hive_table","state":"ACTIVE"}},"traitNames":[],"traits":{}}]},"traitNames":[],"traits":{}}],"type":"ENTITY_FULL_UPDATE","user":"hiveuser"}
> {noformat}
> UI shnapshot: https://monosnap.com/file/Fc91xXDaIT0amrEIpuY8isSAhJ02gK



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message