Mailing-List: contact dev-help@atlas.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@atlas.apache.org
Content-Type: multipart/alternative;
 boundary="===============4697778293896933868=="
MIME-Version: 1.0
Subject: Re: Review Request 61274: ATLAS-1944 - Fix for
 ConcurrentModificationException Exception in HookConsumer (KafkaConsumer)
 while stopping Atlas server
From: Nixon Rodrigues <nixon.rodrigues@freestoneinfotech.com>
To: Apoorv Naik <naik.apoorv@gmail.com>, Madhan Neethiraj <madhan@apache.org>,
 Ashutosh Mestry <amestry@hortonworks.com>,
 Sarath Subramanian <sarath@apache.org>
Cc: atlas <dev@atlas.incubator.apache.org>,
 Nixon Rodrigues <nixon.rodrigues@freestoneinfotech.com>
Date: Wed, 02 Aug 2017 08:46:15 -0000
Message-ID: <20170802084615.16836.14547@reviews-vm2.apache.org>
Auto-Submitted: auto-generated
Sender: Nixon Rodrigues <noreply@reviews.apache.org>
References: <20170801162535.55967.90593@reviews-vm2.apache.org>
In-Reply-To: <20170801162535.55967.90593@reviews-vm2.apache.org>
Reply-To: Nixon Rodrigues <nixon.rodrigues@freestoneinfotech.com>
archived-at: Wed, 02 Aug 2017 08:46:22 -0000

--===============4697778293896933868==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On Aug. 1, 2017, 4:25 p.m., Apoorv Naik wrote:
> > webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
> > Lines 221 (patched)
> > <https://reviews.apache.org/r/61274/diff/1/?file=1786354#file1786354line222>
> >
> >     LOG.info => LOG.debug

Apoorv, this is logged one time and can be helpfull to know if consumer has started, so i am keeping it.


- Nixon


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61274/#review181900
-----------------------------------------------------------


On Aug. 1, 2017, 11:07 a.m., Nixon Rodrigues wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61274/
> -----------------------------------------------------------
> 
> (Updated Aug. 1, 2017, 11:07 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Bugs: ATLAS-1944
>     https://issues.apache.org/jira/browse/ATLAS-1944
> 
> 
> Repository: atlas
> 
> 
> Description
> -------
> 
> Background:- NotificationHookConsumer was throwing ConcurrentModificationException exception while shutting down the consumer thread. The stop method of thread was calling the kafkacosumer.stop method while in run method of HookConsumer which is busy in cosuming the kafka message by polling kafka server. This simaltanous action of stoping and polling causes ConcurrentModificationException.
> 
> Fix:- 
> 
> * Added try finally block in run method which wraps the while loop and when loop ends the finally blocks calls the consumer close method is called which ensures that close is called after consuming is ended. 
> 
> * Replaced the ShutdownThread interface which was delaying the shutdown process of consumer with Runnable interface used earlier.
> 
> 
> Reference : - https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html
> 
> 
> Diffs
> -----
> 
>   notification/src/main/java/org/apache/atlas/kafka/AtlasKafkaConsumer.java d431176 
>   notification/src/main/java/org/apache/atlas/notification/NotificationConsumer.java 0bd75e1 
>   notification/src/test/java/org/apache/atlas/notification/AbstractNotificationConsumerTest.java bcee00c 
>   webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java b8255b3 
> 
> 
> Diff: https://reviews.apache.org/r/61274/diff/1/
> 
> 
> Testing
> -------
> 
> Tested the Atlas kafkaconsumer from hive hook and shutdown of atlas server to check if there is issue while shutdown process.
> Unit test and integration test are passing.
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>


--===============4697778293896933868==--