atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Mestry <ames...@hortonworks.com>
Subject Re: Review Request 60816: Addresses Use of KafkaConsumer in Thread by Implementing Kafka's ShutdownableThread
Date Thu, 13 Jul 2017 05:07:31 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60816/
-----------------------------------------------------------

(Updated July 13, 2017, 5:07 a.m.)


Review request for atlas, Ayub Pathan, Madhan Neethiraj, and Nixon Rodrigues.


Changes
-------

Updates include:
- Added reference section.


Bugs: ATLAS-1944
    https://issues.apache.org/jira/browse/ATLAS-1944


Repository: atlas


Description (updated)
-------

**Background**
Based on discussion with Kafka team member(s), turns out that _KafkaConsumer_ is not thread-safe.
This causes probelem during service shutdown as noted in the ATLAS-1944.

**Solution**
Kafa provides _ShutdownableThread_ which implements thread-safe way for doing shutdown to
_KafaConsumer_ as used in _AtlasKafkaConsumer_.

**References**
- [ShutdownableThread.java](https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/util/ShutdownableThread.java)
- [Usage sample]( https://github.com/apache/kafka/blob/cb0325d484b957432048dd29419f0fa59c5f132d/core/src/main/scala/kafka/log/LogCleaner.scala#L138)


Diffs
-----

  webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java 9e5b864e



Diff: https://reviews.apache.org/r/60816/diff/1/


Testing
-------

**Unit tests**
Executed exisiting tests.

**Functional tests**
- Created entities via Hive.


Thanks,

Ashutosh Mestry


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message