atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tom Beerbower" <tbeerbo...@hortonworks.com>
Subject Review Request 39157: Configure Consumer Groups for Notifications
Date Fri, 09 Oct 2015 01:40:45 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/39157/
-----------------------------------------------------------

Review request for atlas, John Speidel and Shwetha GS.


Bugs: ATLAS-215
    https://issues.apache.org/jira/browse/ATLAS-215


Repository: atlas


Description
-------

Currently with the notification framework we set the properties so that all consumers are
created in a single group...

        //todo take group id as argument to allow multiple consumers??
        properties.put(ConsumerConfig.GROUP_ID_CONFIG, ATLAS_GROUP);

Having Kafka consumers for both the HOOK and ENTITIES topics in a single group causes this
issue ...

    java.lang.IllegalArgumentException: requirement failed: Round-robin assignment is allowed
only if all consumers in the group subscribe to the same topics, AND if the stream counts
across topics are identical for a given consumer instance.
    Topic ATLAS_ENTITIES has the following available consumer streams: Set(atlas_c6401.ambari.apache.org-1443714948705-e518b690-0)
    Topic ATLAS_HOOK has the following available consumer streams: Set(atlas_c6401.ambari.apache.org-1443714214755-8fd9a5f8-0)

We shouldn't have HOOK consumers and ENTITIES consumers in the same group.


Diffs
-----

  notification/pom.xml 2e12520 
  notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 7b3cf89 
  notification/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 29194a4

  notification/src/main/java/org/apache/atlas/notification/NotificationInterface.java d4be07b

  notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java 735655c 

Diff: https://reviews.apache.org/r/39157/diff/


Testing
-------

new unit test added

mvn clean test

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.246 s
[INFO] Finished at: 2015-10-08T21:37:55-04:00
[INFO] Final Memory: 40M/605M
[INFO] ------------------------------------------------------------------------


Thanks,

Tom Beerbower


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message