felix-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carsten Ziegeler <cziege...@apache.org>
Subject Re: Is EventAdmin-1.4.6 really loosing events?
Date Wed, 29 Nov 2017 15:02:42 GMT
Hi,

actually I think this is more or less ok. The events are posted from the
same thread and event admin requires them to be delivered in the order
they are posted. So if one listener is blocking, then delivery of all
other events initiated from the same thread are blocked.

Regards

Carsten


Carsten Ziegeler wrote
> Hi Erwin,
> 
> ah sorry, right :) I was reading modulo 10 instead of equals 10...
> 
> Yes, you're absolutely right in this case, that should not happen.
> 
> This looks like a problem in postEvent somewhere
> 
> Carsten
> 
> 
> Erwin Hogeweg wrote
>> Hi Carsten,
>>
>> Sorry, I think didn’t make myself clear.
>>
>> I am only replacing ONE listener with a BlockingListener, and then I am blocking
ONE event delivery. So I do expect to loose ONE event, not 70k…
>>
>> Or am I missing something?
>>
>> Erwin
>>
>>
>>     @Test
>>     public void testEventing() throws Exception {
>>         // this.addListener(PREFIX + "/0", null);
>>         this.addBlockingListener(PREFIX + "/0", null);
>>         this.addListener(PREFIX + "/1", null);
>>         this.addListener(PREFIX + "/2", null);
>> ...
>>
>>
>>
>> On Nov 29, 2017, at 09:21, Carsten Ziegeler <cziegeler@apache.org<mailto:cziegeler@apache.org>>
wrote:
>>
>> Thanks
>>
>> I'm not sure if event admin could/should do anything in this case. Each
>> blocking listener takes away one thread from the thread pool. As soon as
>> you have as many blocking listeners as the pool has threads, event admin
>> will not deliver any event anymore
>>
>> Regards
>>
>> Carsten
>>
>>
>> Erwin Hogeweg wrote
>> Hi Carsten,
>>
>> After analyzing a bunch of log files we found some evidence that suggested that an
EventAdminThread might be blocked.
>>
>> I was able to duplicate it with a JUnit test. I created a BlockingListener with extends
Listener and override the handleEvent() method. As you can see below we are loosing almost
70000 events because of this one block.
>>
>> I know, eventHandlers are not supposed to block, so we definitely have a design issue
here, but I figured I share the results with you because I am not sure this is what’s expected.
>>
>>
>> Kind Regards,
>>
>> Erwin
>>
>>
>> @Override
>> public void handleEvent(final Event event) {
>> this.test.handleEvent(event, this.payload);
>> count++;
>> if (count == 10) {
>> logger.info<http://logger.info>("{} is blocking forever", Thread.currentThread().getName());
>> while (true) {
>> try {
>> Thread.sleep(60000);
>> } catch (InterruptedException e) {
>> //
>> }
>> }
>> }
>>
>> }
>>
>>
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Starting eventing test StressTestIT
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Preparing test with 15 threads
and 10000 events per thread.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Expecting 1050000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started test with 15 threads
and 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 0 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 0 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : [org.apache.felix.eventadmin.ittests.StressTestIT]
: Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : [org.apache.felix.eventadmin.ittests.StressTestIT]
: [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestITStarted thread
>> Started thread
>> ] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Started thread
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 12 events so far.
>> [org.apache.felix.eventadmin.ittests.BlockingListener] : EventAdminThread #6 is blocking
forever
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Send 10000 events.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far.
>> [org.apache.felix.eventadmin.ittests.StressTestIT] : Received 980007 events so far.
>> …
>>
>>
>>
>> On Nov 15, 2017, at 21:04, Carsten Ziegeler <cziegeler@apache.org<mailto:cziegeler@apache.org><mailto:cziegeler@apache.org>>
wrote:
>>
>> Hi,
>>
>> we haven't experienced anything like this so far and I wouldn't know if
>> any better out of the box way to trouble shoot. I guess the best would
>> be to add additional logging to event admin, so you can follow what's
>> happening inside event admin.
>>
>> If it is somehow reproducible, then switching from postEvent to
>> sendEvent could also help in identifying whether there is a problem in
>> the postEvent handling (which is more complicated than sendEvent).
>>
>> Regards
>>
>> Carsten
>>
>>
>> Erwin Hogeweg wrote
>> Hi,
>>
>> I have a really bizarre problem. It looks like the eventAdmin is intermittent but
frequently loosing events. I find this very hard to believe but the evidence seems to support
my suspicion.
>>
>> I spit out a log msgs just before the event is posted and also when the event is
handled. There is only one eventHandler for this topic, and sure enough every now and again
I see and event be posted but it never comes out at the other end.
>>
>> I have the event admin log level set to 4, but there aren’t any msgs to indicate
that something wrong.
>>
>> I am really at a loss here. Any suggestions for further trouble shooting would be
greatly appreciated.
>>
>> BTW… this seems to happen (and gradually increase) after the system has been up
for several weeks which points in the direction of a resource leak, but I don’t see that
either.
>>
>>
>> Regards,
>>
>> Erwin
>>
>> Java8
>> EventAdmin-1.4.6
>> Equinox: 3.10.2.v20150203-1939
>>
>> Producer side:
>> ...
>>   if (eventAdmin != null) {
>>       if (LOG.isDebugEnabled()) {
>>           LOG.debug("Posting: " + newEvent);
>>       }
>>       eventAdmin.postEvent(newEvent);
>>   } else {
>>       LOG.error("eventAdmin is null.");
>>   }
>>
>>
>> Consumer side:
>>   @Override
>>   public void handleEvent( Event event ) {
>>   LOG.info<http://LOG.info><http://LOG.info><http://LOG.info>("handleEvent:
" + event);
>> ...
>>
>> Erwin Hogeweg
>> CTO
>> 3690 Airport Road
>> Boca Raton, FL 33431
>> P. +1 (954) 556-6565
>> M. +1 (561) 306-7395
>> F. +1 (561) 948-2730
>> [Seecago]<http://www.seecago.com>
>>
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org<mailto:cziegeler@apache.org><mailto:cziegeler@apache.org>
>>
>> Erwin Hogeweg
>> CTO
>> 3690 Airport Road
>> Boca Raton, FL 33431
>> P. +1 (954) 556-6565
>> M. +1 (561) 306-7395
>> F. +1 (561) 948-2730
>> [Seecago]<http://www.seecago.com>
>>
>> --
>> Carsten Ziegeler
>> Adobe Research Switzerland
>> cziegeler@apache.org<mailto:cziegeler@apache.org>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>> For additional commands, e-mail: users-help@felix.apache.org
>>
>>
>> Erwin Hogeweg
>> CTO
>> 3690 Airport Road
>> Boca Raton, FL 33431
>> P. +1 (954) 556-6565
>> M. +1 (561) 306-7395
>> F. +1 (561) 948-2730
>> [Seecago]<http://www.seecago.com>
>>
-- 
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org


Mime
View raw message