felix-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carsten Ziegeler <cziege...@apache.org>
Subject Re: EventAdmin Performance
Date Tue, 10 Apr 2012 12:49:18 GMT
Ok, we had some snowflakes  but they didn't make it down to the ground....

Anyway, I committed a first version (not really polished) to my
sandbox. It runs a parallel test with some threads etc. and by looking
at the log one can compare the time difference between event admin
versions.

Carsten

2012/4/5 Carsten Ziegeler <cziegeler@apache.org>:
> Both is possible :)
>
> 2012/4/5 Marcel Offermans <marcel.offermans@luminis.nl>:
>> Snow? :) I feel for the easter bunnies ;)
>>
>> Thanks!
>>
>> Greetings, Marcel
>>
>>
>> On Apr 5, 2012, at 13:03 , Carsten Ziegeler wrote:
>>
>>> The code is too ugly to make it publically available :)
>>> But if we have snow at Easter (which might really happen), i might
>>> have some time to look into this. Anyway, promised: i'll commit
>>> something in the next two weeks
>>>
>>> Carsten
>>>
>>> 2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
>>>> How about just putting those bundles in your sandbox? I would be interested
to have something to measure the performance of EventAdmin. It does not necessarily need to
be a Pax Exam compatible test.
>>>>
>>>> Greetings, Marcel
>>>>
>>>>
>>>> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote:
>>>>
>>>>> I've a performance test (and several others checking for deadlocks,
>>>>> event delivery etc). They all come as bundles which I deploy into a
>>>>> running instance and then check the results. I guess these could be
>>>>> converted to pax exam tests - but right now I don't have time to do
>>>>> that :(
>>>>>
>>>>> Regards
>>>>> Carsten
>>>>>
>>>>> 2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
>>>>>> Out of curiosity, did anybody ever write a "test" that we can run
to benchmark the performance of EventAdmin. I don't mean an ad-hoc test but something we can
put in the Felix repository (in a sandbox).
>>>>>>
>>>>>> Greetings, Marcel
>>>>>>
>>>>>> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote:
>>>>>>
>>>>>>> Hi Joel,
>>>>>>>
>>>>>>> yes, we noticed some performance problems as well - apart from
the problem
>>>>>>> you mentioned in our case it was a very high load on the service
registry
>>>>>>> (which in addition could also cause deadlocks in the framework).
>>>>>>> So I reimplemented the whole thing (FELIX-3321). You can find
the code now
>>>>>>> in the latest trunk, so you could just build the version from
trunk and see
>>>>>>> how it performs. It basically changes the way how event handlers
are
>>>>>>> fetched from the registry and how the event topic is compared
- filters are
>>>>>>> now only used if really required, the main checks are done by
simple string
>>>>>>> comparisons.
>>>>>>>
>>>>>>> For our really simple performance test case with more than 1
million
>>>>>>> events, it took more than 30 seconds with the old implementation,
the new
>>>>>>> one processes the same amount in less than a second.
>>>>>>>
>>>>>>> Not sure about your problem if events getting eaten up. With
the new impl
>>>>>>> it should be much easier to track this down (if it still happens)
>>>>>>>
>>>>>>> Regards
>>>>>>> Carsten
>>>>>>>
>>>>>>> 2012/4/4 Joel Schuster <joel.schuster@gmx.com>
>>>>>>>
>>>>>>>> Carsten et al.,****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> I've been running some profiling against some test components
using
>>>>>>>> EventAdmin. I'm seeing some really degenerate behavior that's
causing some
>>>>>>>> real slowdowns for message delivery coming from a few places.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Environment: running 400+ messages per second and with 15+
topics with
>>>>>>>> various levels of hierarchy.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Because of the general ‘brute force’ means by which
>>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList
>>>>>>>> works, and how the multiple layers of
>>>>>>>> org.apache.felix.framework.capabilityset.CapabilitySet.match
work when
>>>>>>>> topics have multiple layers of hierarchy with the names I
can double the
>>>>>>>> throughput of EventAdmin by simply changing my topic names
to one layer of
>>>>>>>> hierarchy. Of course I lose the advantages of the fiters,
but if I can
>>>>>>>> double the throughput then it seems worth it.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Even then it’s still taking more than half a millisecond
(optimally,
>>>>>>>> sometimes many tens of milliseconds) for any particular message
to make its
>>>>>>>> way through the EventAdmin system, being pushed and handled.
That seems
>>>>>>>> like a long time. Part of the overhead comes from the
>>>>>>>> org.apache.felix.framework.ServiceRegistry.getService and
ungetService
>>>>>>>> calls that happen for every single message push.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> It seems to me that some sort of caching of previous search
results in
>>>>>>>> each of these domains would be well worth the effort.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Carsten, I noticed that you had commented about an improved
EventAdmin in
>>>>>>>> the sandbox. Can you please point that out to me, I’d like
to try it out.*
>>>>>>>> ***
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Also, I’ve notice that the EventAdmin is not delivering
all the events
>>>>>>>> that are pushed to a topic. Small percentages (1 / 1000)
are simply being
>>>>>>>> eaten up somewhere… I can’t seem to track down any reason
or log for why
>>>>>>>> the events simply disappeared.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> Thanks all for consideration.****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> ****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> - Joel****
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>> ** **
>>>>>>>>
>>>>>>>>> -----Original Message-----
>>>>>>>>
>>>>>>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
>>>>>>>>
>>>>>>>>> Sent: Tuesday, January 24, 2012 2:55 AM
>>>>>>>>
>>>>>>>>> To: users@felix.apache.org
>>>>>>>>
>>>>>>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> we haven't implemented this feature yet but will do once
the final
>>>>>>>>
>>>>>>>>> spec is publically available.
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> I've written a new implementation of the event admin
which is in the
>>>>>>>>
>>>>>>>>> felix sandbox atm, this one is significanlty faster than
the old
>>>>>>>>
>>>>>>>>> implementation from trunk.
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>
>>>>>>>>> Carsten
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> 2012/1/18 DBinSD <cdiebner@gmail.com>:
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>> Thanks Holger, this is exactly the info I was looking
for.  It sounds
>>>>>>>> like
>>>>>>>>
>>>>>>>>>> the Async Unordered Events might get me the performance
I was looking
>>>>>>>>
>>>>>>>>> for
>>>>>>>>
>>>>>>>>>> (though the implications of 'unordered' events might
be a problem in
>>>>>>>> some
>>>>>>>>
>>>>>>>>>> applications).  I will keep an eye out for an implementation
of this
>>>>>>>>
>>>>>>>>>> feature.
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>> Thanks again
>>>>>>>>
>>>>>>>>>> Chris
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>> Holger Hoffstätte-4 wrote:
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> On 18.01.2012 03:35, DBinSD wrote:
>>>>>>>>
>>>>>>>>>>>> Does the felix implementation of the EventAdmin
service support the
>>>>>>>>
>>>>>>>>>>>> delivery
>>>>>>>>
>>>>>>>>>>>> of events, asynchronously and in parallel
to multiple listeners at
>>>>>>>> once?
>>>>>>>>
>>>>>>>>>>>> It
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> This was addressed as part of RFC 157 ("Updates
to EventAdmin")
>>>>>>>>
>>>>>>>>> sometime
>>>>>>>>
>>>>>>>>>>> in 2010 and accepted, though I don't remember
offhand what the
>>>>>>>>
>>>>>>>>>>> implementation status was/is. IIRC it was part
of 4.3.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> The solution added a set of constants that express
basic delivery
>>>>>>>>
>>>>>>>>>>> behaviour/expectations. Not really enforceable
end-to-end QoS
>>>>>>>>
>>>>>>>>> constraints
>>>>>>>>
>>>>>>>>>>> (which are impossible in Java anyway) but better
than nothing.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> --snip--
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> 5.2    Asynchronous Unordered Events
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> The following constants are added to EventConstants.
This allows an
>>>>>>>>
>>>>>>>>> Event
>>>>>>>>
>>>>>>>>>>> Handler to indicate that it is willing to relax
the event ordering
>>>>>>>>
>>>>>>>>>>> requirements so that Event Admin can optimize
delivery.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> String EVENT_DELIVERY = "event.delivery"
>>>>>>>>
>>>>>>>>>>> Service Registration property specifying the
delivery qualities
>>>>>>>> requested
>>>>>>>>
>>>>>>>>>>> by an Event Handler service.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> Event handlers MAY be registered with this property.
Each value of
>>>>>>>> this
>>>>>>>>
>>>>>>>>>>> property is a string specifying a delivery quality
for the Event
>>>>>>>> handler.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> The value of this property must be of type String,
String[], or
>>>>>>>>
>>>>>>>>>>> Collection<String>.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> Since:
>>>>>>>>
>>>>>>>>>>>        1.3
>>>>>>>>
>>>>>>>>>>> See Also:
>>>>>>>>
>>>>>>>>>>>        DELIVERY_ASYNC_ORDERED
>>>>>>>>
>>>>>>>>>>>        DELIVERY_ASYNC_UNORDERED
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> String DELIVERY_ASYNC_ORDERED = "async.ordered"
>>>>>>>>
>>>>>>>>>>> Event Handler delivery quality value specifying
the Event Handler
>>>>>>>> requires
>>>>>>>>
>>>>>>>>>>> asynchronously delivered events be delivered
in order. Ordered
>>>>>>>> delivery
>>>>>>>>
>>>>>>>>> is
>>>>>>>>
>>>>>>>>>>> the default for asynchronously delivered events.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> This delivery quality value is mutually exclusive
with
>>>>>>>>
>>>>>>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both this
value and
>>>>>>>>
>>>>>>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for an
event handler, this
>>>>>>>>
>>>>>>>>> value
>>>>>>>>
>>>>>>>>>>> takes precedence.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> Since:
>>>>>>>>
>>>>>>>>>>>        1.3
>>>>>>>>
>>>>>>>>>>> See Also:
>>>>>>>>
>>>>>>>>>>>        EVENT_DELIVERY
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> String DELIVERY_ASYNC_UNORDERED = "async.unordered"
>>>>>>>>
>>>>>>>>>>> Event Handler delivery quality value specifying
the Event Handler does
>>>>>>>>
>>>>>>>>> not
>>>>>>>>
>>>>>>>>>>> require asynchronously delivered events be delivered
in order. This
>>>>>>>> may
>>>>>>>>
>>>>>>>>>>> allow an Event Admin implementation to optimize
asynchronous event
>>>>>>>>
>>>>>>>>>>> delivery by relaxing ordering requirements.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> This delivery quality value is mutually exclusive
with
>>>>>>>>
>>>>>>>>>>> DELIVERY_ASYNC_ORDERED. However, if both this
value and
>>>>>>>>
>>>>>>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an event
handler,
>>>>>>>>
>>>>>>>>>>> DELIVERY_ASYNC_ORDERED takes precedence.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> Since:
>>>>>>>>
>>>>>>>>>>>        1.3
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> See Also:
>>>>>>>>
>>>>>>>>>>>        EVENT_DELIVERY
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> --snip--
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> Also see
>>>>>>>>
>>>>>>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi-
>>>>>>>>
>>>>>>>>> eventadmin-and-when-not
>>>>>>>>
>>>>>>>>>>> for BJ's official answer.
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> hope this helps
>>>>>>>>
>>>>>>>>>>> Holger
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>>>>
>>>>>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>> --
>>>>>>>>
>>>>>>>>>> View this message in context: http://old.nabble.com/EventAdmin-
>>>>>>>>
>>>>>>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html
>>>>>>>>
>>>>>>>>>> Sent from the Apache Felix - Users mailing list archive
at Nabble.com.
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>>>>
>>>>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> --
>>>>>>>>
>>>>>>>>> Carsten Ziegeler
>>>>>>>>
>>>>>>>>> cziegeler@apache.org
>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>
>>>>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>>>>
>>>>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Carsten Ziegeler
>>>>>>> cziegeler@apache.org
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Carsten Ziegeler
>>>>> cziegeler@apache.org
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>>> For additional commands, e-mail: users-help@felix.apache.org
>>>>
>>>
>>>
>>>
>>> --
>>> Carsten Ziegeler
>>> cziegeler@apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>>> For additional commands, e-mail: users-help@felix.apache.org
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
>> For additional commands, e-mail: users-help@felix.apache.org
>>
>
>
>
> --
> Carsten Ziegeler
> cziegeler@apache.org



-- 
Carsten Ziegeler
cziegeler@apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org


Mime
View raw message