atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sarath Subramanian (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (ATLAS-1145) Storm hook with kafka topic and hive bolt picks wrong hive table name
Date Tue, 20 Sep 2016 17:13:21 GMT

    [ https://issues.apache.org/jira/browse/ATLAS-1145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15507148#comment-15507148
] 

Sarath Subramanian edited comment on ATLAS-1145 at 9/20/16 5:12 PM:
--------------------------------------------------------------------

Thanks for the samples [~shwethags], [~yhemanth] and [~ayubkhan]. I was able to create a storm
topology with kafka spout/hive bolt using the samples from Ayub's link. I'm not able to reproduce
this issue. Below is the kafka topic entity definition created by the storm topology, the
kafka topic entity in atlas gets assigned the right topic name and not the hive table name.


{
  "requestId": "qtp1448525331-188 - f7904223-bac3-489b-b593-ffbb3100debd",
  "definition": {
    "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
    "id": {
      "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
      "id": "5669f2ba-8397-44c5-bf0d-0544533979a4",
      "version": 0,
      "typeName": "kafka_topic",
      "state": "ACTIVE"
    },
    "typeName": "kafka_topic",
    "values": {
      "name": "kafka_topic_7",
      "description": null,
      "uri": "localhost:2181",
      "qualifiedName": "kafka_topic_7@ErieCluster",
      "topic": "kafka_topic_7",
      "owner": "storm"
    },
    "traitNames": [],
    "traits": {}
  }
}


was (Author: sarath.kum4r@gmail.com):
Thanks for the samples [~shwethags], [~yhemanth] and [~ayubkhan]. I was able to create a storm
topology with kafka spout/hive bolt using the samples from Ayub's link. I'm not able to reproduce
this issue. Attaching the entity definition from REST, the kafka topic entity in atlas gets
assigned the right topic name and not the hive table name. Attaching the kafka topic entity
definition created by the storm topology.

{
  "requestId": "qtp1448525331-188 - f7904223-bac3-489b-b593-ffbb3100debd",
  "definition": {
    "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
    "id": {
      "jsonClass": "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
      "id": "5669f2ba-8397-44c5-bf0d-0544533979a4",
      "version": 0,
      "typeName": "kafka_topic",
      "state": "ACTIVE"
    },
    "typeName": "kafka_topic",
    "values": {
      "name": "kafka_topic_7",
      "description": null,
      "uri": "localhost:2181",
      "qualifiedName": "kafka_topic_7@ErieCluster",
      "topic": "kafka_topic_7",
      "owner": "storm"
    },
    "traitNames": [],
    "traits": {}
  }
}

> Storm hook with kafka topic and hive bolt picks wrong hive table name 
> ----------------------------------------------------------------------
>
>                 Key: ATLAS-1145
>                 URL: https://issues.apache.org/jira/browse/ATLAS-1145
>             Project: Atlas
>          Issue Type: Bug
>            Reporter: Shwetha G S
>            Assignee: Sarath Subramanian
>
> Storm hook with kafka topic and hive bolt picks kafka topic name as hive table name



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message