felix-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carsten Ziegeler <cziege...@apache.org>
Subject Re: EventAdmin Performance
Date Thu, 05 Apr 2012 11:03:58 GMT
The code is too ugly to make it publically available :)
But if we have snow at Easter (which might really happen), i might
have some time to look into this. Anyway, promised: i'll commit
something in the next two weeks

Carsten

2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
> How about just putting those bundles in your sandbox? I would be interested to have something
to measure the performance of EventAdmin. It does not necessarily need to be a Pax Exam compatible
test.
>
> Greetings, Marcel
>
>
> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote:
>
>> I've a performance test (and several others checking for deadlocks,
>> event delivery etc). They all come as bundles which I deploy into a
>> running instance and then check the results. I guess these could be
>> converted to pax exam tests - but right now I don't have time to do
>> that :(
>>
>> Regards
>> Carsten
>>
>> 2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
>>> Out of curiosity, did anybody ever write a "test" that we can run to benchmark
the performance of EventAdmin. I don't mean an ad-hoc test but something we can put in the
Felix repository (in a sandbox).
>>>
>>> Greetings, Marcel
>>>
>>> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote:
>>>
>>>> Hi Joel,
>>>>
>>>> yes, we noticed some performance problems as well - apart from the problem
>>>> you mentioned in our case it was a very high load on the service registry
>>>> (which in addition could also cause deadlocks in the framework).
>>>> So I reimplemented the whole thing (FELIX-3321). You can find the code now
>>>> in the latest trunk, so you could just build the version from trunk and see
>>>> how it performs. It basically changes the way how event handlers are
>>>> fetched from the registry and how the event topic is compared - filters are
>>>> now only used if really required, the main checks are done by simple string
>>>> comparisons.
>>>>
>>>> For our really simple performance test case with more than 1 million
>>>> events, it took more than 30 seconds with the old implementation, the new
>>>> one processes the same amount in less than a second.
>>>>
>>>> Not sure about your problem if events getting eaten up. With the new impl
>>>> it should be much easier to track this down (if it still happens)
>>>>
>>>> Regards
>>>> Carsten
>>>>
>>>> 2012/4/4 Joel Schuster <joel.schuster@gmx.com>
>>>>
>>>>> Carsten et al.,****
>>>>>
>>>>> ** **
>>>>>
>>>>> I've been running some profiling against some test components using
>>>>> EventAdmin. I'm seeing some really degenerate behavior that's causing
some
>>>>> real slowdowns for message delivery coming from a few places.****
>>>>>
>>>>> ** **
>>>>>
>>>>> Environment: running 400+ messages per second and with 15+ topics with
>>>>> various levels of hierarchy.****
>>>>>
>>>>> ** **
>>>>>
>>>>> Because of the general ‘brute force’ means by which
>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList
>>>>> works, and how the multiple layers of
>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.match work when
>>>>> topics have multiple layers of hierarchy with the names I can double
the
>>>>> throughput of EventAdmin by simply changing my topic names to one layer
of
>>>>> hierarchy. Of course I lose the advantages of the fiters, but if I can
>>>>> double the throughput then it seems worth it.****
>>>>>
>>>>> ** **
>>>>>
>>>>> Even then it’s still taking more than half a millisecond (optimally,
>>>>> sometimes many tens of milliseconds) for any particular message to make
its
>>>>> way through the EventAdmin system, being pushed and handled. That seems
>>>>> like a long time. Part of the overhead comes from the
>>>>> org.apache.felix.framework.ServiceRegistry.getService and ungetService
>>>>> calls that happen for every single message push.****
>>>>>
>>>>> ** **
>>>>>
>>>>> It seems to me that some sort of caching of previous search results in
>>>>> each of these domains would be well worth the effort.****
>>>>>
>>>>> ** **
>>>>>
>>>>> Carsten, I noticed that you had commented about an improved EventAdmin
in
>>>>> the sandbox. Can you please point that out to me, I’d like to try it
out.*
>>>>> ***
>>>>>
>>>>> ** **
>>>>>
>>>>> Also, I’ve notice that the EventAdmin is not delivering all the events
>>>>> that are pushed to a topic. Small percentages (1 / 1000) are simply being
>>>>> eaten up somewhere… I can’t seem to track down any reason or log
for why
>>>>> the events simply disappeared.****
>>>>>
>>>>> ** **
>>>>>
>>>>> ** **
>>>>>
>>>>> Thanks all for consideration.****
>>>>>
>>>>> ** **
>>>>>
>>>>> ** **
>>>>>
>>>>> ** **
>>>>>
>>>>> ****
>>>>>
>>>>> ** **
>>>>>
>>>>> - Joel****
>>>>>
>>>>> ** **
>>>>>
>>>>> ** **
>>>>>
>>>>>> -----Original Message-----
>>>>>
>>>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>>>>
>>>>>> Sent: Tuesday, January 24, 2012 2:55 AM
>>>>>
>>>>>> To: users@felix.apache.org
>>>>>
>>>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler
>>>>>
>>>>>>
>>>>>
>>>>>> Hi,
>>>>>
>>>>>>
>>>>>
>>>>>> we haven't implemented this feature yet but will do once the final
>>>>>
>>>>>> spec is publically available.
>>>>>
>>>>>>
>>>>>
>>>>>> I've written a new implementation of the event admin which is in
the
>>>>>
>>>>>> felix sandbox atm, this one is significanlty faster than the old
>>>>>
>>>>>> implementation from trunk.
>>>>>
>>>>>>
>>>>>
>>>>>> Regards
>>>>>
>>>>>> Carsten
>>>>>
>>>>>>
>>>>>
>>>>>> 2012/1/18 DBinSD <cdiebner@gmail.com>:
>>>>>
>>>>>>>
>>>>>
>>>>>>> Thanks Holger, this is exactly the info I was looking for.  It
sounds
>>>>> like
>>>>>
>>>>>>> the Async Unordered Events might get me the performance I was
looking
>>>>>
>>>>>> for
>>>>>
>>>>>>> (though the implications of 'unordered' events might be a problem
in
>>>>> some
>>>>>
>>>>>>> applications).  I will keep an eye out for an implementation
of this
>>>>>
>>>>>>> feature.
>>>>>
>>>>>>>
>>>>>
>>>>>>> Thanks again
>>>>>
>>>>>>> Chris
>>>>>
>>>>>>>
>>>>>
>>>>>>>
>>>>>
>>>>>>>
>>>>>
>>>>>>> Holger Hoffstätte-4 wrote:
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> On 18.01.2012 03:35, DBinSD wrote:
>>>>>
>>>>>>>>> Does the felix implementation of the EventAdmin service
support the
>>>>>
>>>>>>>>> delivery
>>>>>
>>>>>>>>> of events, asynchronously and in parallel to multiple
listeners at
>>>>> once?
>>>>>
>>>>>>>>> It
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> This was addressed as part of RFC 157 ("Updates to EventAdmin")
>>>>>
>>>>>> sometime
>>>>>
>>>>>>>> in 2010 and accepted, though I don't remember offhand what
the
>>>>>
>>>>>>>> implementation status was/is. IIRC it was part of 4.3.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> The solution added a set of constants that express basic
delivery
>>>>>
>>>>>>>> behaviour/expectations. Not really enforceable end-to-end
QoS
>>>>>
>>>>>> constraints
>>>>>
>>>>>>>> (which are impossible in Java anyway) but better than nothing.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> --snip--
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> 5.2    Asynchronous Unordered Events
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> The following constants are added to EventConstants. This
allows an
>>>>>
>>>>>> Event
>>>>>
>>>>>>>> Handler to indicate that it is willing to relax the event
ordering
>>>>>
>>>>>>>> requirements so that Event Admin can optimize delivery.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> String EVENT_DELIVERY = "event.delivery"
>>>>>
>>>>>>>> Service Registration property specifying the delivery qualities
>>>>> requested
>>>>>
>>>>>>>> by an Event Handler service.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> Event handlers MAY be registered with this property. Each
value of
>>>>> this
>>>>>
>>>>>>>> property is a string specifying a delivery quality for the
Event
>>>>> handler.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> The value of this property must be of type String, String[],
or
>>>>>
>>>>>>>> Collection<String>.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> Since:
>>>>>
>>>>>>>>        1.3
>>>>>
>>>>>>>> See Also:
>>>>>
>>>>>>>>        DELIVERY_ASYNC_ORDERED
>>>>>
>>>>>>>>        DELIVERY_ASYNC_UNORDERED
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> String DELIVERY_ASYNC_ORDERED = "async.ordered"
>>>>>
>>>>>>>> Event Handler delivery quality value specifying the Event
Handler
>>>>> requires
>>>>>
>>>>>>>> asynchronously delivered events be delivered in order. Ordered
>>>>> delivery
>>>>>
>>>>>> is
>>>>>
>>>>>>>> the default for asynchronously delivered events.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> This delivery quality value is mutually exclusive with
>>>>>
>>>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both this value and
>>>>>
>>>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for an event handler,
this
>>>>>
>>>>>> value
>>>>>
>>>>>>>> takes precedence.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> Since:
>>>>>
>>>>>>>>        1.3
>>>>>
>>>>>>>> See Also:
>>>>>
>>>>>>>>        EVENT_DELIVERY
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> String DELIVERY_ASYNC_UNORDERED = "async.unordered"
>>>>>
>>>>>>>> Event Handler delivery quality value specifying the Event
Handler does
>>>>>
>>>>>> not
>>>>>
>>>>>>>> require asynchronously delivered events be delivered in order.
This
>>>>> may
>>>>>
>>>>>>>> allow an Event Admin implementation to optimize asynchronous
event
>>>>>
>>>>>>>> delivery by relaxing ordering requirements.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> This delivery quality value is mutually exclusive with
>>>>>
>>>>>>>> DELIVERY_ASYNC_ORDERED. However, if both this value and
>>>>>
>>>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an event handler,
>>>>>
>>>>>>>> DELIVERY_ASYNC_ORDERED takes precedence.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> Since:
>>>>>
>>>>>>>>        1.3
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> See Also:
>>>>>
>>>>>>>>        EVENT_DELIVERY
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> --snip--
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> Also see
>>>>>
>>>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi-
>>>>>
>>>>>> eventadmin-and-when-not
>>>>>
>>>>>>>> for BJ's official answer.
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> hope this helps
>>>>>
>>>>>>>> Holger
>>>>>
>>>>>>>>
>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>
>>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>
>>>>>>>>
>>>>>
>>>>>>>>
>>>>>
>>>>>>>>
>>>>>
>>>>>>>
>>>>>
>>>>>>> --
>>>>>
>>>>>>> View this message in context: http://old.nabble.com/EventAdmin-
>>>>>
>>>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html
>>>>>
>>>>>>> Sent from the Apache Felix - Users mailing list archive at Nabble.com.
>>>>>
>>>>>>>
>>>>>
>>>>>>>
>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>
>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>
>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>
>>>>>>>
>>>>>
>>>>>>
>>>>>
>>>>>>
>>>>>
>>>>>>
>>>>>
>>>>>> --
>>>>>
>>>>>> Carsten Ziegeler
>>>>>
>>>>>> cziegeler@apache.org
>>>>>
>>>>>>
>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>
>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>
>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Carsten Ziegeler
>>>> cziegeler@apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>> For additional commands, e-mail: users-help@felix.apache.org
>>>
>>
>>
>>
>> --
>> Carsten Ziegeler
>> cziegeler@apache.org
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>> For additional commands, e-mail: users-help@felix.apache.org
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> For additional commands, e-mail: users-help@felix.apache.org
>



-- 
Carsten Ziegeler
cziegeler@apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org


Mime
View raw message