atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shwetha GS" <sshivalingamur...@hortonworks.com>
Subject Re: Review Request 39157: Configure Consumer Groups for Notifications
Date Mon, 12 Oct 2015 03:52:22 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39157/#review102204
-----------------------------------------------------------



notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java (line 332)
<https://reviews.apache.org/r/39157/#comment159765>

    Instead of the if condition, add property in application.properties for atlas.kafka.hook.group.id.
In general group id should be read from atlas.kafka.<topicname>.group.id


- Shwetha GS


On Oct. 12, 2015, 2:22 a.m., Tom Beerbower wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/39157/
> -----------------------------------------------------------
> 
> (Updated Oct. 12, 2015, 2:22 a.m.)
> 
> 
> Review request for atlas, John Speidel and Shwetha GS.
> 
> 
> Bugs: ATLAS-215
>     https://issues.apache.org/jira/browse/ATLAS-215
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Currently with the notification framework we set the properties so that all consumers
are created in a single group...
> 
>         //todo take group id as argument to allow multiple consumers??
>         properties.put(ConsumerConfig.GROUP_ID_CONFIG, ATLAS_GROUP);
> 
> Having Kafka consumers for both the HOOK and ENTITIES topics in a single group causes
this issue ...
> 
>     java.lang.IllegalArgumentException: requirement failed: Round-robin assignment is
allowed only if all consumers in the group subscribe to the same topics, AND if the stream
counts across topics are identical for a given consumer instance.
>     Topic ATLAS_ENTITIES has the following available consumer streams: Set(atlas_c6401.ambari.apache.org-1443714948705-e518b690-0)
>     Topic ATLAS_HOOK has the following available consumer streams: Set(atlas_c6401.ambari.apache.org-1443714214755-8fd9a5f8-0)
> 
> We shouldn't have HOOK consumers and ENTITIES consumers in the same group.
> 
> 
> Diffs
> -----
> 
>   notification/pom.xml 2e12520 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 7b3cf89 
>   notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java 735655c

> 
> Diff: https://reviews.apache.org/r/39157/diff/
> 
> 
> Testing
> -------
> 
> new unit test added
> 
> mvn clean test
> 
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 12.246 s
> [INFO] Finished at: 2015-10-08T21:37:55-04:00
> [INFO] Final Memory: 40M/605M
> [INFO] ------------------------------------------------------------------------
> 
> 
> Thanks,
> 
> Tom Beerbower
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message