felix-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Schuster" <joel.schus...@gmx.com>
Subject RE: EventAdmin Performance
Date Wed, 04 Apr 2012 20:20:50 GMT
All,

Thanks for the input. I'm trying out the 1.2.15-SNAPSHOT version of event
admin and it's behaving much better. Thanks!

I am still getting dropped events; now to look into that.

- Joel


> -----Original Message-----
> From: Marcel Offermans [mailto:marcel.offermans@luminis.nl]
> Sent: Wednesday, April 04, 2012 12:40 PM
> To: users@felix.apache.org
> Subject: Re: EventAdmin Performance
> 
> How about just putting those bundles in your sandbox? I would be
> interested to have something to measure the performance of EventAdmin.
> It does not necessarily need to be a Pax Exam compatible test.
> 
> Greetings, Marcel
> 
> 
> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote:
> 
> > I've a performance test (and several others checking for deadlocks,
> > event delivery etc). They all come as bundles which I deploy into a
> > running instance and then check the results. I guess these could be
> > converted to pax exam tests - but right now I don't have time to do
> > that :(
> >
> > Regards
> > Carsten
> >
> > 2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
> >> Out of curiosity, did anybody ever write a "test" that we can run to
> benchmark the performance of EventAdmin. I don't mean an ad-hoc test but
> something we can put in the Felix repository (in a sandbox).
> >>
> >> Greetings, Marcel
> >>
> >> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote:
> >>
> >>> Hi Joel,
> >>>
> >>> yes, we noticed some performance problems as well - apart from the
> problem
> >>> you mentioned in our case it was a very high load on the service
registry
> >>> (which in addition could also cause deadlocks in the framework).
> >>> So I reimplemented the whole thing (FELIX-3321). You can find the code
> now
> >>> in the latest trunk, so you could just build the version from trunk
and see
> >>> how it performs. It basically changes the way how event handlers are
> >>> fetched from the registry and how the event topic is compared -
filters
> are
> >>> now only used if really required, the main checks are done by simple
> string
> >>> comparisons.
> >>>
> >>> For our really simple performance test case with more than 1 million
> >>> events, it took more than 30 seconds with the old implementation, the
> new
> >>> one processes the same amount in less than a second.
> >>>
> >>> Not sure about your problem if events getting eaten up. With the new
> impl
> >>> it should be much easier to track this down (if it still happens)
> >>>
> >>> Regards
> >>> Carsten
> >>>
> >>> 2012/4/4 Joel Schuster <joel.schuster@gmx.com>
> >>>
> >>>> Carsten et al.,****
> >>>>
> >>>> ** **
> >>>>
> >>>> I've been running some profiling against some test components using
> >>>> EventAdmin. I'm seeing some really degenerate behavior that's causing
> some
> >>>> real slowdowns for message delivery coming from a few places.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Environment: running 400+ messages per second and with 15+ topics
> with
> >>>> various levels of hierarchy.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Because of the general ‘brute force’ means by which
> >>>>
> org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList
> >>>> works, and how the multiple layers of
> >>>> org.apache.felix.framework.capabilityset.CapabilitySet.match work
> when
> >>>> topics have multiple layers of hierarchy with the names I can double
> the
> >>>> throughput of EventAdmin by simply changing my topic names to one
> layer of
> >>>> hierarchy. Of course I lose the advantages of the fiters, but if I
can
> >>>> double the throughput then it seems worth it.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Even then it’s still taking more than half a millisecond (optimally,
> >>>> sometimes many tens of milliseconds) for any particular message to
> make its
> >>>> way through the EventAdmin system, being pushed and handled. That
> seems
> >>>> like a long time. Part of the overhead comes from the
> >>>> org.apache.felix.framework.ServiceRegistry.getService and
> ungetService
> >>>> calls that happen for every single message push.****
> >>>>
> >>>> ** **
> >>>>
> >>>> It seems to me that some sort of caching of previous search results
in
> >>>> each of these domains would be well worth the effort.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Carsten, I noticed that you had commented about an improved
> EventAdmin in
> >>>> the sandbox. Can you please point that out to me, I’d like to try it
out.*
> >>>> ***
> >>>>
> >>>> ** **
> >>>>
> >>>> Also, I’ve notice that the EventAdmin is not delivering all the
events
> >>>> that are pushed to a topic. Small percentages (1 / 1000) are simply
> being
> >>>> eaten up somewhere… I can’t seem to track down any reason or log for
> why
> >>>> the events simply disappeared.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> Thanks all for consideration.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> - Joel****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>>> -----Original Message-----
> >>>>
> >>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
> >>>>
> >>>>> Sent: Tuesday, January 24, 2012 2:55 AM
> >>>>
> >>>>> To: users@felix.apache.org
> >>>>
> >>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler
> >>>>
> >>>>>
> >>>>
> >>>>> Hi,
> >>>>
> >>>>>
> >>>>
> >>>>> we haven't implemented this feature yet but will do once the final
> >>>>
> >>>>> spec is publically available.
> >>>>
> >>>>>
> >>>>
> >>>>> I've written a new implementation of the event admin which is in
the
> >>>>
> >>>>> felix sandbox atm, this one is significanlty faster than the old
> >>>>
> >>>>> implementation from trunk.
> >>>>
> >>>>>
> >>>>
> >>>>> Regards
> >>>>
> >>>>> Carsten
> >>>>
> >>>>>
> >>>>
> >>>>> 2012/1/18 DBinSD <cdiebner@gmail.com>:
> >>>>
> >>>>>>
> >>>>
> >>>>>> Thanks Holger, this is exactly the info I was looking for. 
It
sounds
> >>>> like
> >>>>
> >>>>>> the Async Unordered Events might get me the performance I was
> looking
> >>>>
> >>>>> for
> >>>>
> >>>>>> (though the implications of 'unordered' events might be a problem
> in
> >>>> some
> >>>>
> >>>>>> applications).  I will keep an eye out for an implementation
of
this
> >>>>
> >>>>>> feature.
> >>>>
> >>>>>>
> >>>>
> >>>>>> Thanks again
> >>>>
> >>>>>> Chris
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>> Holger Hoffstätte-4 wrote:
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> On 18.01.2012 03:35, DBinSD wrote:
> >>>>
> >>>>>>>> Does the felix implementation of the EventAdmin service
support
> the
> >>>>
> >>>>>>>> delivery
> >>>>
> >>>>>>>> of events, asynchronously and in parallel to multiple
listeners
at
> >>>> once?
> >>>>
> >>>>>>>> It
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This was addressed as part of RFC 157 ("Updates to EventAdmin")
> >>>>
> >>>>> sometime
> >>>>
> >>>>>>> in 2010 and accepted, though I don't remember offhand what
the
> >>>>
> >>>>>>> implementation status was/is. IIRC it was part of 4.3.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The solution added a set of constants that express basic
delivery
> >>>>
> >>>>>>> behaviour/expectations. Not really enforceable end-to-end
QoS
> >>>>
> >>>>> constraints
> >>>>
> >>>>>>> (which are impossible in Java anyway) but better than nothing.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> --snip--
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> 5.2    Asynchronous Unordered Events
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The following constants are added to EventConstants. This
allows
> an
> >>>>
> >>>>> Event
> >>>>
> >>>>>>> Handler to indicate that it is willing to relax the event
ordering
> >>>>
> >>>>>>> requirements so that Event Admin can optimize delivery.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String EVENT_DELIVERY = "event.delivery"
> >>>>
> >>>>>>> Service Registration property specifying the delivery qualities
> >>>> requested
> >>>>
> >>>>>>> by an Event Handler service.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Event handlers MAY be registered with this property. Each
value of
> >>>> this
> >>>>
> >>>>>>> property is a string specifying a delivery quality for the
Event
> >>>> handler.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The value of this property must be of type String, String[],
or
> >>>>
> >>>>>>> Collection<String>.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        DELIVERY_ASYNC_ORDERED
> >>>>
> >>>>>>>        DELIVERY_ASYNC_UNORDERED
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String DELIVERY_ASYNC_ORDERED = "async.ordered"
> >>>>
> >>>>>>> Event Handler delivery quality value specifying the Event
Handler
> >>>> requires
> >>>>
> >>>>>>> asynchronously delivered events be delivered in order. Ordered
> >>>> delivery
> >>>>
> >>>>> is
> >>>>
> >>>>>>> the default for asynchronously delivered events.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This delivery quality value is mutually exclusive with
> >>>>
> >>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both this value and
> >>>>
> >>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for an event handler,
> this
> >>>>
> >>>>> value
> >>>>
> >>>>>>> takes precedence.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        EVENT_DELIVERY
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String DELIVERY_ASYNC_UNORDERED = "async.unordered"
> >>>>
> >>>>>>> Event Handler delivery quality value specifying the Event
Handler
> does
> >>>>
> >>>>> not
> >>>>
> >>>>>>> require asynchronously delivered events be delivered in
order.
This
> >>>> may
> >>>>
> >>>>>>> allow an Event Admin implementation to optimize asynchronous
> event
> >>>>
> >>>>>>> delivery by relaxing ordering requirements.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This delivery quality value is mutually exclusive with
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED. However, if both this value and
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an event handler,
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED takes precedence.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        EVENT_DELIVERY
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> --snip--
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Also see
> >>>>
> >>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi-
> >>>>
> >>>>> eventadmin-and-when-not
> >>>>
> >>>>>>> for BJ's official answer.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> hope this helps
> >>>>
> >>>>>>> Holger
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
---------------------------------------------------------------------
> >>>>
> >>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>> --
> >>>>
> >>>>>> View this message in context: http://old.nabble.com/EventAdmin-
> >>>>
> >>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html
> >>>>
> >>>>>> Sent from the Apache Felix - Users mailing list archive at
> Nabble.com.
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>>
---------------------------------------------------------------------
> >>>>
> >>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>>>>
> >>>>
> >>>>>
> >>>>
> >>>>>
> >>>>
> >>>>>
> >>>>
> >>>>> --
> >>>>
> >>>>> Carsten Ziegeler
> >>>>
> >>>>> cziegeler@apache.org
> >>>>
> >>>>>
> >>>>
> >>>>>
---------------------------------------------------------------------
> >>>>
> >>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Carsten Ziegeler
> >>> cziegeler@apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >> For additional commands, e-mail: users-help@felix.apache.org
> >>
> >
> >
> >
> > --
> > Carsten Ziegeler
> > cziegeler@apache.org
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> > For additional commands, e-mail: users-help@felix.apache.org
> >
> >
> >
> >
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> For additional commands, e-mail: users-help@felix.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org


Mime
View raw message