atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Madhan Neethiraj <mad...@apache.org>
Subject Re: Review Request 60597: ATLAS-1908 : - Updating old Kafka consumer api properties to reflect change in new KafkaConsumer configs for Atlas
Date Fri, 14 Jul 2017 22:29:28 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60597/#review180596
-----------------------------------------------------------


Ship it!




Ship It!

- Madhan Neethiraj


On July 14, 2017, 9:52 a.m., Nixon Rodrigues wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60597/
> -----------------------------------------------------------
> 
> (Updated July 14, 2017, 9:52 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1908
>     https://issues.apache.org/jira/browse/ATLAS-1908
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> This patch handles old kafka consumer configs values to be added as new configs if new
configs are not present , so that Kafka consumer starts gracefully even if new configs are
not added explicitly.
> 
> eg. atlas.kafka.auto.commit.enable=false to atlas.kafka.enable.auto.commit=false
>     atlas.kafka.auto.offset.reset=smallest to atlas.kafka.auto.offset.reset=earliest
>     atlas.kafka.session.timeout.ms=30000
>     
>     Also added poll.timeout.ms in configs so kafka consumer can be configurable.This
involve change in AtlasKafkaConsumer receive interface.
> 
> 
> Diffs
> -----
> 
>   distro/src/conf/atlas-application.properties c3213df 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java 9c15243 
>   notification/src/main/java/org/apache/atlas/kafka/KafkaNotification.java 366c8a7 
>   notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java
22e40f9 
>   notification/src/test/java/org/apache/atlas/kafka/KafkaConsumerTest.java 70059cb 
>   notification/src/test/java/org/apache/atlas/kafka/KafkaNotificationTest.java c791d43

>   notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java
8324b57 
>   webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 0dea0e2

>   webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
650ca0a 
>   webapp/src/test/java/org/apache/atlas/web/integration/BaseResourceIT.java c036cfa 
> 
> 
> Diff: https://reviews.apache.org/r/60597/diff/2/
> 
> 
> Testing
> -------
> 
> Executed UT/IT's using mvn clean install.
> Tested KafkaConsumer with Hive Hook by creating table in hive.
> Atlas starts gracefully and Kafka consumer is up without error when new properties are
not added.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message