atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nixon Rodrigues <nixon.rodrig...@freestoneinfotech.com>
Subject Review Request 60597: ATLAS-1908 : - Updating old Kafka consumer api properties to reflect change in new KafkaConsumer configs for Atlas
Date Mon, 03 Jul 2017 13:17:48 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60597/
-----------------------------------------------------------

Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.


Bugs: ATLAS-1908
    https://issues.apache.org/jira/browse/ATLAS-1908


Repository: atlas


Description
-------

This patch handles old kafka consumer configs values to be added as new configs if new configs
are not present , so that Kafka consumer starts gracefully even if new configs are not added
explicitly.

eg. atlas.kafka.auto.commit.enable=false to atlas.kafka.enable.auto.commit=false
    atlas.kafka.auto.offset.reset=smallest to atlas.kafka.auto.offset.reset=earliest
    atlas.kafka.session.timeout.ms=30000
    
    Also added poll.timeout.ms in configs so kafka consumer can be configurable.This involve
change in AtlasKafkaConsumer receive interface.


Diffs
-----

  distro/src/conf/atlas-application.properties 474f253 
  notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 9c15243 
  notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 366c8a7 
  notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java 22e40f9

  notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java 70059cb 
  notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java c791d43 
  notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
8324b57 
  webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 9e5b864

  webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
650ca0a 
  webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java c036cfa 


Diff: https://reviews.apache.org/r/60597/diff/1/


Testing
-------

Executed UT/IT's using mvn clean install.
Tested KafkaConsumer with Hive Hook by creating table in hive.
Atlas starts gracefully and Kafka consumer is up without error when new properties are not
added.


Thanks,

Nixon Rodrigues


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message