flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ahmed Vila <av...@devlogic.eu>
Subject Flume Sink, sometimes transaction not being commited/rolledback & closed
Date Tue, 09 Sep 2014 14:51:57 GMT
Hi Group,

We've implemented custom Flume Sink.
We do some DB lookups for sanity checks within it and bunch of stuff in
order to slightly prepare data for insertion into the postgresql via copy

It gets the channel, transaction and it executes begin() method.
Of course, there is try-catch-finally block wrapper that ensures commit or
rollback methods are called and finally transaction is closed.

Sometimes it happens that there is an exception due to transaction.begin
called before transaction.commit or rollback.

I don't see it possible unless there are two threads take the same

Another possible reason might be short code in the catch block that could
prevent call to the transaction.rollback and transaction.close.
However, it doesn't throw an exception within catch block as it would
emerge in logs.

This is relevant part of the code:
public Status process() throws EventDeliveryException {
Channel channel = getChannel();
Transaction transaction = channel.getTransaction();
try {
event = channel.take();
} catch (Exception ex) {
LOG.error("Problem with sink", ex);
LOG.error("Exception: " + ex.toString());

LOG.info("Start source...");

} finally {

return result;
}// end method

Thanks in advance


Best regards,
Ahmed Vila

This e-mail and any attachment is for authorised use by the intended 
recipient(s) only. This email contains confidential information. It should 
not be copied, disclosed to, retained or used by, any party other than the 
intended recipient. Any unauthorised distribution, dissemination or copying 
of this E-mail or its attachments, and/or any use of any information 
contained in them, is strictly prohibited and may be illegal. If you are 
not an intended recipient then please promptly delete this e-mail and any 
attachment and all copies and inform the sender directly via email. Any 
emails that you send to us may be monitored by systems or persons other 
than the named communicant for the purposes of ascertaining whether the 
communication complies with the law and company policies.

View raw message