atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom Beerbower" <>
Subject Review Request 39157: Configure Consumer Groups for Notifications
Date Fri, 09 Oct 2015 01:40:45 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for atlas, John Speidel and Shwetha GS.

Bugs: ATLAS-215

Repository: atlas


Currently with the notification framework we set the properties so that all consumers are
created in a single group...

        //todo take group id as argument to allow multiple consumers??
        properties.put(ConsumerConfig.GROUP_ID_CONFIG, ATLAS_GROUP);

Having Kafka consumers for both the HOOK and ENTITIES topics in a single group causes this
issue ...

    java.lang.IllegalArgumentException: requirement failed: Round-robin assignment is allowed
only if all consumers in the group subscribe to the same topics, AND if the stream counts
across topics are identical for a given consumer instance.
    Topic ATLAS_ENTITIES has the following available consumer streams: Set(
    Topic ATLAS_HOOK has the following available consumer streams: Set(

We shouldn't have HOOK consumers and ENTITIES consumers in the same group.


  notification/pom.xml 2e12520 
  notification/src/main/java/org/apache/atlas/kafka/ 7b3cf89 
  notification/src/main/java/org/apache/atlas/notification/ 29194a4

  notification/src/main/java/org/apache/atlas/notification/ d4be07b

  notification/src/test/java/org/apache/atlas/kafka/ 735655c 



new unit test added

mvn clean test

[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.246 s
[INFO] Finished at: 2015-10-08T21:37:55-04:00
[INFO] Final Memory: 40M/605M
[INFO] ------------------------------------------------------------------------


Tom Beerbower

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message