felix-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcel Offermans <marcel.offerm...@luminis.nl>
Subject Re: EventAdmin Performance
Date Tue, 10 Apr 2012 12:54:55 GMT
Thanks Carsten! I'll play around with it this evening! No snow here either btw, just rain.
;)

On Apr 10, 2012, at 14:49 PM, Carsten Ziegeler wrote:

> Ok, we had some snowflakes  but they didn't make it down to the ground....
> 
> Anyway, I committed a first version (not really polished) to my
> sandbox. It runs a parallel test with some threads etc. and by looking
> at the log one can compare the time difference between event admin
> versions.
> 
> Carsten
> 
> 2012/4/5 Carsten Ziegeler <cziegeler@apache.org>:
>> Both is possible :)
>> 
>> 2012/4/5 Marcel Offermans <marcel.offermans@luminis.nl>:
>>> Snow? :) I feel for the easter bunnies ;)
>>> 
>>> Thanks!
>>> 
>>> Greetings, Marcel
>>> 
>>> 
>>> On Apr 5, 2012, at 13:03 , Carsten Ziegeler wrote:
>>> 
>>>> The code is too ugly to make it publically available :)
>>>> But if we have snow at Easter (which might really happen), i might
>>>> have some time to look into this. Anyway, promised: i'll commit
>>>> something in the next two weeks
>>>> 
>>>> Carsten
>>>> 
>>>> 2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
>>>>> How about just putting those bundles in your sandbox? I would be interested
to have something to measure the performance of EventAdmin. It does not necessarily need to
be a Pax Exam compatible test.
>>>>> 
>>>>> Greetings, Marcel
>>>>> 
>>>>> 
>>>>> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote:
>>>>> 
>>>>>> I've a performance test (and several others checking for deadlocks,
>>>>>> event delivery etc). They all come as bundles which I deploy into
a
>>>>>> running instance and then check the results. I guess these could
be
>>>>>> converted to pax exam tests - but right now I don't have time to
do
>>>>>> that :(
>>>>>> 
>>>>>> Regards
>>>>>> Carsten
>>>>>> 
>>>>>> 2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
>>>>>>> Out of curiosity, did anybody ever write a "test" that we can
run to benchmark the performance of EventAdmin. I don't mean an ad-hoc test but something
we can put in the Felix repository (in a sandbox).
>>>>>>> 
>>>>>>> Greetings, Marcel
>>>>>>> 
>>>>>>> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote:
>>>>>>> 
>>>>>>>> Hi Joel,
>>>>>>>> 
>>>>>>>> yes, we noticed some performance problems as well - apart
from the problem
>>>>>>>> you mentioned in our case it was a very high load on the
service registry
>>>>>>>> (which in addition could also cause deadlocks in the framework).
>>>>>>>> So I reimplemented the whole thing (FELIX-3321). You can
find the code now
>>>>>>>> in the latest trunk, so you could just build the version
from trunk and see
>>>>>>>> how it performs. It basically changes the way how event handlers
are
>>>>>>>> fetched from the registry and how the event topic is compared
- filters are
>>>>>>>> now only used if really required, the main checks are done
by simple string
>>>>>>>> comparisons.
>>>>>>>> 
>>>>>>>> For our really simple performance test case with more than
1 million
>>>>>>>> events, it took more than 30 seconds with the old implementation,
the new
>>>>>>>> one processes the same amount in less than a second.
>>>>>>>> 
>>>>>>>> Not sure about your problem if events getting eaten up. With
the new impl
>>>>>>>> it should be much easier to track this down (if it still
happens)
>>>>>>>> 
>>>>>>>> Regards
>>>>>>>> Carsten
>>>>>>>> 
>>>>>>>> 2012/4/4 Joel Schuster <joel.schuster@gmx.com>
>>>>>>>> 
>>>>>>>>> Carsten et al.,****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> I've been running some profiling against some test components
using
>>>>>>>>> EventAdmin. I'm seeing some really degenerate behavior
that's causing some
>>>>>>>>> real slowdowns for message delivery coming from a few
places.****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> Environment: running 400+ messages per second and with
15+ topics with
>>>>>>>>> various levels of hierarchy.****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> Because of the general ‘brute force’ means by which
>>>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList
>>>>>>>>> works, and how the multiple layers of
>>>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.match
work when
>>>>>>>>> topics have multiple layers of hierarchy with the names
I can double the
>>>>>>>>> throughput of EventAdmin by simply changing my topic
names to one layer of
>>>>>>>>> hierarchy. Of course I lose the advantages of the fiters,
but if I can
>>>>>>>>> double the throughput then it seems worth it.****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> Even then it’s still taking more than half a millisecond
(optimally,
>>>>>>>>> sometimes many tens of milliseconds) for any particular
message to make its
>>>>>>>>> way through the EventAdmin system, being pushed and handled.
That seems
>>>>>>>>> like a long time. Part of the overhead comes from the
>>>>>>>>> org.apache.felix.framework.ServiceRegistry.getService
and ungetService
>>>>>>>>> calls that happen for every single message push.****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> It seems to me that some sort of caching of previous
search results in
>>>>>>>>> each of these domains would be well worth the effort.****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> Carsten, I noticed that you had commented about an improved
EventAdmin in
>>>>>>>>> the sandbox. Can you please point that out to me, I’d
like to try it out.*
>>>>>>>>> ***
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> Also, I’ve notice that the EventAdmin is not delivering
all the events
>>>>>>>>> that are pushed to a topic. Small percentages (1 / 1000)
are simply being
>>>>>>>>> eaten up somewhere… I can’t seem to track down any
reason or log for why
>>>>>>>>> the events simply disappeared.****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> Thanks all for consideration.****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> ****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> - Joel****
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>> ** **
>>>>>>>>> 
>>>>>>>>>> -----Original Message-----
>>>>>>>>> 
>>>>>>>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>>>>>>>> 
>>>>>>>>>> Sent: Tuesday, January 24, 2012 2:55 AM
>>>>>>>>> 
>>>>>>>>>> To: users@felix.apache.org
>>>>>>>>> 
>>>>>>>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> Hi,
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> we haven't implemented this feature yet but will
do once the final
>>>>>>>>> 
>>>>>>>>>> spec is publically available.
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> I've written a new implementation of the event admin
which is in the
>>>>>>>>> 
>>>>>>>>>> felix sandbox atm, this one is significanlty faster
than the old
>>>>>>>>> 
>>>>>>>>>> implementation from trunk.
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> Regards
>>>>>>>>> 
>>>>>>>>>> Carsten
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 2012/1/18 DBinSD <cdiebner@gmail.com>:
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> Thanks Holger, this is exactly the info I was
looking for.  It sounds
>>>>>>>>> like
>>>>>>>>> 
>>>>>>>>>>> the Async Unordered Events might get me the performance
I was looking
>>>>>>>>> 
>>>>>>>>>> for
>>>>>>>>> 
>>>>>>>>>>> (though the implications of 'unordered' events
might be a problem in
>>>>>>>>> some
>>>>>>>>> 
>>>>>>>>>>> applications).  I will keep an eye out for an
implementation of this
>>>>>>>>> 
>>>>>>>>>>> feature.
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> Thanks again
>>>>>>>>> 
>>>>>>>>>>> Chris
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> Holger Hoffstätte-4 wrote:
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> On 18.01.2012 03:35, DBinSD wrote:
>>>>>>>>> 
>>>>>>>>>>>>> Does the felix implementation of the
EventAdmin service support the
>>>>>>>>> 
>>>>>>>>>>>>> delivery
>>>>>>>>> 
>>>>>>>>>>>>> of events, asynchronously and in parallel
to multiple listeners at
>>>>>>>>> once?
>>>>>>>>> 
>>>>>>>>>>>>> It
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> This was addressed as part of RFC 157 ("Updates
to EventAdmin")
>>>>>>>>> 
>>>>>>>>>> sometime
>>>>>>>>> 
>>>>>>>>>>>> in 2010 and accepted, though I don't remember
offhand what the
>>>>>>>>> 
>>>>>>>>>>>> implementation status was/is. IIRC it was
part of 4.3.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> The solution added a set of constants that
express basic delivery
>>>>>>>>> 
>>>>>>>>>>>> behaviour/expectations. Not really enforceable
end-to-end QoS
>>>>>>>>> 
>>>>>>>>>> constraints
>>>>>>>>> 
>>>>>>>>>>>> (which are impossible in Java anyway) but
better than nothing.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> --snip--
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> 5.2    Asynchronous Unordered Events
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> The following constants are added to EventConstants.
This allows an
>>>>>>>>> 
>>>>>>>>>> Event
>>>>>>>>> 
>>>>>>>>>>>> Handler to indicate that it is willing to
relax the event ordering
>>>>>>>>> 
>>>>>>>>>>>> requirements so that Event Admin can optimize
delivery.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> String EVENT_DELIVERY = "event.delivery"
>>>>>>>>> 
>>>>>>>>>>>> Service Registration property specifying
the delivery qualities
>>>>>>>>> requested
>>>>>>>>> 
>>>>>>>>>>>> by an Event Handler service.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> Event handlers MAY be registered with this
property. Each value of
>>>>>>>>> this
>>>>>>>>> 
>>>>>>>>>>>> property is a string specifying a delivery
quality for the Event
>>>>>>>>> handler.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> The value of this property must be of type
String, String[], or
>>>>>>>>> 
>>>>>>>>>>>> Collection<String>.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> Since:
>>>>>>>>> 
>>>>>>>>>>>>        1.3
>>>>>>>>> 
>>>>>>>>>>>> See Also:
>>>>>>>>> 
>>>>>>>>>>>>        DELIVERY_ASYNC_ORDERED
>>>>>>>>> 
>>>>>>>>>>>>        DELIVERY_ASYNC_UNORDERED
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> String DELIVERY_ASYNC_ORDERED = "async.ordered"
>>>>>>>>> 
>>>>>>>>>>>> Event Handler delivery quality value specifying
the Event Handler
>>>>>>>>> requires
>>>>>>>>> 
>>>>>>>>>>>> asynchronously delivered events be delivered
in order. Ordered
>>>>>>>>> delivery
>>>>>>>>> 
>>>>>>>>>> is
>>>>>>>>> 
>>>>>>>>>>>> the default for asynchronously delivered
events.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> This delivery quality value is mutually exclusive
with
>>>>>>>>> 
>>>>>>>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both
this value and
>>>>>>>>> 
>>>>>>>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for
an event handler, this
>>>>>>>>> 
>>>>>>>>>> value
>>>>>>>>> 
>>>>>>>>>>>> takes precedence.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> Since:
>>>>>>>>> 
>>>>>>>>>>>>        1.3
>>>>>>>>> 
>>>>>>>>>>>> See Also:
>>>>>>>>> 
>>>>>>>>>>>>        EVENT_DELIVERY
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> String DELIVERY_ASYNC_UNORDERED = "async.unordered"
>>>>>>>>> 
>>>>>>>>>>>> Event Handler delivery quality value specifying
the Event Handler does
>>>>>>>>> 
>>>>>>>>>> not
>>>>>>>>> 
>>>>>>>>>>>> require asynchronously delivered events be
delivered in order. This
>>>>>>>>> may
>>>>>>>>> 
>>>>>>>>>>>> allow an Event Admin implementation to optimize
asynchronous event
>>>>>>>>> 
>>>>>>>>>>>> delivery by relaxing ordering requirements.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> This delivery quality value is mutually exclusive
with
>>>>>>>>> 
>>>>>>>>>>>> DELIVERY_ASYNC_ORDERED. However, if both
this value and
>>>>>>>>> 
>>>>>>>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an
event handler,
>>>>>>>>> 
>>>>>>>>>>>> DELIVERY_ASYNC_ORDERED takes precedence.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> Since:
>>>>>>>>> 
>>>>>>>>>>>>        1.3
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> See Also:
>>>>>>>>> 
>>>>>>>>>>>>        EVENT_DELIVERY
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> --snip--
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> Also see
>>>>>>>>> 
>>>>>>>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi-
>>>>>>>>> 
>>>>>>>>>> eventadmin-and-when-not
>>>>>>>>> 
>>>>>>>>>>>> for BJ's official answer.
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> hope this helps
>>>>>>>>> 
>>>>>>>>>>>> Holger
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> 
>>>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>>>>> 
>>>>>>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>> 
>>>>>>>>>>> View this message in context: http://old.nabble.com/EventAdmin-
>>>>>>>>> 
>>>>>>>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html
>>>>>>>>> 
>>>>>>>>>>> Sent from the Apache Felix - Users mailing list
archive at Nabble.com.
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> 
>>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>>>>> 
>>>>>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>> 
>>>>>>>>>> Carsten Ziegeler
>>>>>>>>> 
>>>>>>>>>> cziegeler@apache.org
>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> 
>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>>>>> 
>>>>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Carsten Ziegeler
>>>>>>>> cziegeler@apache.org
>>>>>>> 
>>>>>>> 
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Carsten Ziegeler
>>>>>> cziegeler@apache.org
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Carsten Ziegeler
>>>> cziegeler@apache.org
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>> For additional commands, e-mail: users-help@felix.apache.org
>>> 
>> 
>> 
>> 
>> --
>> Carsten Ziegeler
>> cziegeler@apache.org
> 
> 
> 
> -- 
> Carsten Ziegeler
> cziegeler@apache.org
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> For additional commands, e-mail: users-help@felix.apache.org
> 
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org


Mime
View raw message