flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roshan Naik <ros...@hortonworks.com>
Subject Re: Flume Sink, sometimes transaction not being commited/rolledback & closed
Date Fri, 12 Sep 2014 02:57:37 GMT
you could try putting a conditional breakpoint which triggers only after 2k
events

On Thu, Sep 11, 2014 at 1:20 PM, Ahmed Vila <avila@devlogic.eu> wrote:

> Hi Roshan,
>
> Yeah, I could but I'll probably spent a whole day debugging as there is
> like 2000 events processed before the issue emerges.
> I might just do so as the issue is obviously very specific and no one
> seems experiencing the same.
>
> I'll get back with my findings...
>
>
> On Thu, Sep 11, 2014 at 10:10 PM, Roshan Naik <roshan@hortonworks.com>
> wrote:
>
>> Can you reproduce this within a debugger and observe what is sequence of
>> things that leads to the problem.
>>
>> On Tue, Sep 9, 2014 at 2:34 PM, Ahmed Vila <avila@devlogic.eu> wrote:
>>
>>> maybe I haven't expressed my self in a god way...
>>>
>>> I've already confirmed that flume logs all unhandled exceptions, so it
>>> should log the ones emerging from within that catch block.
>>> Actually, I've mixed something before with that cacheDBPool being null,
>>> so I can confirm that flume does log that exception.
>>>
>>> On Tue, Sep 9, 2014 at 11:31 PM, Ahmed Vila <avila@devlogic.eu> wrote:
>>>
>>>> Hi Roshan,
>>>>
>>>> I'm puzzled because there should be at least a note about exception in
>>>> catch block as I've already confirmed that flume logs those.
>>>>
>>>> Is it possible that SInkRunner just mixes up transactions ?
>>>> Anyone already found himself in similar situation ?
>>>>
>>>>
>>>> On Tue, Sep 9, 2014 at 11:15 PM, Roshan Naik <roshan@hortonworks.com>
>>>> wrote:
>>>>
>>>>> you may want to rollback immediately after the log messages  in the
>>>>> catch block and not at the end.
>>>>>
>>>>> On Tue, Sep 9, 2014 at 7:51 AM, Ahmed Vila <avila@devlogic.eu>
wrote:
>>>>>
>>>>>> Hi Group,
>>>>>>
>>>>>> We've implemented custom Flume Sink.
>>>>>> We do some DB lookups for sanity checks within it and bunch of stuff
>>>>>> in order to slightly prepare data for insertion into the postgresql
via
>>>>>> copy command.
>>>>>>
>>>>>> It gets the channel, transaction and it executes begin() method.
>>>>>> Of course, there is try-catch-finally block wrapper that ensures
>>>>>> commit or rollback methods are called and finally transaction is
closed.
>>>>>>
>>>>>> Sometimes it happens that there is an exception due to
>>>>>> transaction.begin called before transaction.commit or rollback.
>>>>>>
>>>>>> I don't see it possible unless there are two threads take the same
>>>>>> transaction.
>>>>>>
>>>>>> Another possible reason might be short code in the catch block that
>>>>>> could prevent call to the transaction.rollback and transaction.close.
>>>>>> However, it doesn't throw an exception within catch block as it would
>>>>>> emerge in logs.
>>>>>>
>>>>>>
>>>>>>
>>>>>> This is relevant part of the code:
>>>>>> @Override
>>>>>> public Status process() throws EventDeliveryException {
>>>>>> Channel channel = getChannel();
>>>>>> Transaction transaction = channel.getTransaction();
>>>>>> ...
>>>>>> try {
>>>>>> transaction.begin();
>>>>>> event = channel.take();
>>>>>> ...
>>>>>> transaction.commit();
>>>>>> } catch (Exception ex) {
>>>>>> LOG.error("Problem with sink", ex);
>>>>>> LOG.error("Exception: " + ex.toString());
>>>>>>
>>>>>> LOG.info("Start source...");
>>>>>> this.cacheDBPool.clear();
>>>>>> CommandExecutor.execUnStopCmd(this.flumeFolderLogs);
>>>>>>
>>>>>> transaction.rollback();
>>>>>> } finally {
>>>>>> transaction.close();
>>>>>> }
>>>>>>
>>>>>> return result;
>>>>>> }// end method
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks in advance
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> Best regards,
>>>>>> Ahmed Vila
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> This e-mail and any attachment is for authorised use by the intended
>>>>>> recipient(s) only. This email contains confidential information.
It should
>>>>>> not be copied, disclosed to, retained or used by, any party other
than the
>>>>>> intended recipient. Any unauthorised distribution, dissemination
or copying
>>>>>> of this E-mail or its attachments, and/or any use of any information
>>>>>> contained in them, is strictly prohibited and may be illegal. If
you are
>>>>>> not an intended recipient then please promptly delete this e-mail
and any
>>>>>> attachment and all copies and inform the sender directly via email.
Any
>>>>>> emails that you send to us may be monitored by systems or persons
other
>>>>>> than the named communicant for the purposes of ascertaining whether
the
>>>>>> communication complies with the law and company policies.
>>>>>
>>>>>
>>>>>
>>>>> CONFIDENTIALITY NOTICE
>>>>> NOTICE: This message is intended for the use of the individual or
>>>>> entity to which it is addressed and may contain information that is
>>>>> confidential, privileged and exempt from disclosure under applicable
law.
>>>>> If the reader of this message is not the intended recipient, you are
hereby
>>>>> notified that any printing, copying, dissemination, distribution,
>>>>> disclosure or forwarding of this communication is strictly prohibited.
If
>>>>> you have received this communication in error, please contact the sender
>>>>> immediately and delete it from your system. Thank You.
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Best regards,
>>>> Ahmed Vila
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> Best regards,
>>> Ahmed Vila | Senior software developer
>>> DevLogic | Sarajevo | Bosnia and Herzegovina
>>>
>>> Office : +387 33 942 123
>>> Mobile: +387 62 139 348
>>>
>>> Website: www.devlogic.eu
>>> E-mail   : avila@devlogic.eu
>>> ---------------------------------------------------------------------
>>> This e-mail and any attachment is for authorised use by the intended
>>> recipient(s) only. This email contains confidential information. It should
>>> not be copied, disclosed to, retained or used by, any party other than the
>>> intended recipient. Any unauthorised distribution, dissemination or copying
>>> of this E-mail or its attachments, and/or any use of any information
>>> contained in them, is strictly prohibited and may be illegal. If you are
>>> not an intended recipient then please promptly delete this e-mail and any
>>> attachment and all copies and inform the sender directly via email. Any
>>> emails that you send to us may be monitored by systems or persons other
>>> than the named communicant for the purposes of ascertaining whether the
>>> communication complies with the law and company policies.
>>>
>>> ---------------------------------------------------------------------
>>> This e-mail and any attachment is for authorised use by the intended
>>> recipient(s) only. This email contains confidential information. It should
>>> not be copied, disclosed to, retained or used by, any party other than the
>>> intended recipient. Any unauthorised distribution, dissemination or copying
>>> of this E-mail or its attachments, and/or any use of any information
>>> contained in them, is strictly prohibited and may be illegal. If you are
>>> not an intended recipient then please promptly delete this e-mail and any
>>> attachment and all copies and inform the sender directly via email. Any
>>> emails that you send to us may be monitored by systems or persons other
>>> than the named communicant for the purposes of ascertaining whether the
>>> communication complies with the law and company policies.
>>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
> ---------------------------------------------------------------------
> This e-mail and any attachment is for authorised use by the intended
> recipient(s) only. This email contains confidential information. It should
> not be copied, disclosed to, retained or used by, any party other than the
> intended recipient. Any unauthorised distribution, dissemination or copying
> of this E-mail or its attachments, and/or any use of any information
> contained in them, is strictly prohibited and may be illegal. If you are
> not an intended recipient then please promptly delete this e-mail and any
> attachment and all copies and inform the sender directly via email. Any
> emails that you send to us may be monitored by systems or persons other
> than the named communicant for the purposes of ascertaining whether the
> communication complies with the law and company policies.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message