Mailing-List: contact users-help@felix.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@felix.apache.org
Received-SPF: pass (nike.apache.org: domain of joel.schuster@gmx.com
 designates 74.208.5.67 as permitted sender)
From: "Joel Schuster" <joel.schuster@gmx.com>
To: <users@felix.apache.org>
References: <020701cd1286$f5d5a770$e180f650$@gmx.com>
 <CAKkCf4p0xEN9c8C1hcCKXGEb0+oeO5S4mD3SCP6KinZE=i_LJQ@mail.gmail.com>
 <917FBA2D-CB63-4CF1-B3FC-8864CB2604AA@luminis.nl>
 <CAKkCf4oHT_Bhea=JeHNbpD_do+mpun_Pni7JdqL0awyck-qM-Q@mail.gmail.com>
 <20C52308-608C-42E5-ABB8-EB5FC9969970@luminis.nl>
In-Reply-To: <20C52308-608C-42E5-ABB8-EB5FC9969970@luminis.nl>
Subject: RE: EventAdmin Performance
Date: Wed, 4 Apr 2012 14:20:50 -0600
Message-ID: <027b01cd12a0$6eb44e30$4c1cea90$@gmx.com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Thread-Index: AQIfTmlDm+gB+nvlbTvn9199ffPKmAE8pQGgAUr3JOsCLV570AJfXhUOla4A/CA=
Content-Language: en-us

All,

Thanks for the input. I'm trying out the 1.2.15-SNAPSHOT version of =
event
admin and it's behaving much better. Thanks!

I am still getting dropped events; now to look into that.

- Joel


> -----Original Message-----
> From: Marcel Offermans [mailto:marcel.offermans@luminis.nl]
> Sent: Wednesday, April 04, 2012 12:40 PM
> To: users@felix.apache.org
> Subject: Re: EventAdmin Performance
>=20
> How about just putting those bundles in your sandbox? I would be
> interested to have something to measure the performance of EventAdmin.
> It does not necessarily need to be a Pax Exam compatible test.
>=20
> Greetings, Marcel
>=20
>=20
> On Apr 4, 2012, at 19:50 PM, Carsten Ziegeler wrote:
>=20
> > I've a performance test (and several others checking for deadlocks,
> > event delivery etc). They all come as bundles which I deploy into a
> > running instance and then check the results. I guess these could be
> > converted to pax exam tests - but right now I don't have time to do
> > that :(
> >
> > Regards
> > Carsten
> >
> > 2012/4/4 Marcel Offermans <marcel.offermans@luminis.nl>:
> >> Out of curiosity, did anybody ever write a "test" that we can run =
to
> benchmark the performance of EventAdmin. I don't mean an ad-hoc test =
but
> something we can put in the Felix repository (in a sandbox).
> >>
> >> Greetings, Marcel
> >>
> >> On Apr 4, 2012, at 19:30 PM, Carsten Ziegeler wrote:
> >>
> >>> Hi Joel,
> >>>
> >>> yes, we noticed some performance problems as well - apart from the
> problem
> >>> you mentioned in our case it was a very high load on the service
registry
> >>> (which in addition could also cause deadlocks in the framework).
> >>> So I reimplemented the whole thing (FELIX-3321). You can find the =
code
> now
> >>> in the latest trunk, so you could just build the version from =
trunk
and see
> >>> how it performs. It basically changes the way how event handlers =
are
> >>> fetched from the registry and how the event topic is compared -
filters
> are
> >>> now only used if really required, the main checks are done by =
simple
> string
> >>> comparisons.
> >>>
> >>> For our really simple performance test case with more than 1 =
million
> >>> events, it took more than 30 seconds with the old implementation, =
the
> new
> >>> one processes the same amount in less than a second.
> >>>
> >>> Not sure about your problem if events getting eaten up. With the =
new
> impl
> >>> it should be much easier to track this down (if it still happens)
> >>>
> >>> Regards
> >>> Carsten
> >>>
> >>> 2012/4/4 Joel Schuster <joel.schuster@gmx.com>
> >>>
> >>>> Carsten et al.,****
> >>>>
> >>>> ** **
> >>>>
> >>>> I've been running some profiling against some test components =
using
> >>>> EventAdmin. I'm seeing some really degenerate behavior that's =
causing
> some
> >>>> real slowdowns for message delivery coming from a few places.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Environment: running 400+ messages per second and with 15+ topics
> with
> >>>> various levels of hierarchy.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Because of the general =91brute force=92 means by which
> >>>>
> =
org.apache.felix.framework.capabilityset.CapabilitySet.convertArrayToList=

> >>>> works, and how the multiple layers of
> >>>> org.apache.felix.framework.capabilityset.CapabilitySet.match work
> when
> >>>> topics have multiple layers of hierarchy with the names I can =
double
> the
> >>>> throughput of EventAdmin by simply changing my topic names to one
> layer of
> >>>> hierarchy. Of course I lose the advantages of the fiters, but if =
I
can
> >>>> double the throughput then it seems worth it.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Even then it=92s still taking more than half a millisecond =
(optimally,
> >>>> sometimes many tens of milliseconds) for any particular message =
to
> make its
> >>>> way through the EventAdmin system, being pushed and handled. That
> seems
> >>>> like a long time. Part of the overhead comes from the
> >>>> org.apache.felix.framework.ServiceRegistry.getService and
> ungetService
> >>>> calls that happen for every single message push.****
> >>>>
> >>>> ** **
> >>>>
> >>>> It seems to me that some sort of caching of previous search =
results
in
> >>>> each of these domains would be well worth the effort.****
> >>>>
> >>>> ** **
> >>>>
> >>>> Carsten, I noticed that you had commented about an improved
> EventAdmin in
> >>>> the sandbox. Can you please point that out to me, I=92d like to =
try it
out.*
> >>>> ***
> >>>>
> >>>> ** **
> >>>>
> >>>> Also, I=92ve notice that the EventAdmin is not delivering all the
events
> >>>> that are pushed to a topic. Small percentages (1 / 1000) are =
simply
> being
> >>>> eaten up somewhere=85 I can=92t seem to track down any reason or =
log for
> why
> >>>> the events simply disappeared.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> Thanks all for consideration.****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>> ****
> >>>>
> >>>> ** **
> >>>>
> >>>> - Joel****
> >>>>
> >>>> ** **
> >>>>
> >>>> ** **
> >>>>
> >>>>> -----Original Message-----
> >>>>
> >>>>> From: Carsten Ziegeler [mailto:cziegeler@apache.org]
> >>>>
> >>>>> Sent: Tuesday, January 24, 2012 2:55 AM
> >>>>
> >>>>> To: users@felix.apache.org
> >>>>
> >>>>> Subject: Re: EventAdmin Asynchronous/Parallel EventHandler
> >>>>
> >>>>>
> >>>>
> >>>>> Hi,
> >>>>
> >>>>>
> >>>>
> >>>>> we haven't implemented this feature yet but will do once the =
final
> >>>>
> >>>>> spec is publically available.
> >>>>
> >>>>>
> >>>>
> >>>>> I've written a new implementation of the event admin which is in =
the
> >>>>
> >>>>> felix sandbox atm, this one is significanlty faster than the old
> >>>>
> >>>>> implementation from trunk.
> >>>>
> >>>>>
> >>>>
> >>>>> Regards
> >>>>
> >>>>> Carsten
> >>>>
> >>>>>
> >>>>
> >>>>> 2012/1/18 DBinSD <cdiebner@gmail.com>:
> >>>>
> >>>>>>
> >>>>
> >>>>>> Thanks Holger, this is exactly the info I was looking for.  It
sounds
> >>>> like
> >>>>
> >>>>>> the Async Unordered Events might get me the performance I was
> looking
> >>>>
> >>>>> for
> >>>>
> >>>>>> (though the implications of 'unordered' events might be a =
problem
> in
> >>>> some
> >>>>
> >>>>>> applications).  I will keep an eye out for an implementation of
this
> >>>>
> >>>>>> feature.
> >>>>
> >>>>>>
> >>>>
> >>>>>> Thanks again
> >>>>
> >>>>>> Chris
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>> Holger Hoffst=E4tte-4 wrote:
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> On 18.01.2012 03:35, DBinSD wrote:
> >>>>
> >>>>>>>> Does the felix implementation of the EventAdmin service =
support
> the
> >>>>
> >>>>>>>> delivery
> >>>>
> >>>>>>>> of events, asynchronously and in parallel to multiple =
listeners
at
> >>>> once?
> >>>>
> >>>>>>>> It
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This was addressed as part of RFC 157 ("Updates to =
EventAdmin")
> >>>>
> >>>>> sometime
> >>>>
> >>>>>>> in 2010 and accepted, though I don't remember offhand what the
> >>>>
> >>>>>>> implementation status was/is. IIRC it was part of 4.3.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The solution added a set of constants that express basic =
delivery
> >>>>
> >>>>>>> behaviour/expectations. Not really enforceable end-to-end QoS
> >>>>
> >>>>> constraints
> >>>>
> >>>>>>> (which are impossible in Java anyway) but better than nothing.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> --snip--
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> 5.2    Asynchronous Unordered Events
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The following constants are added to EventConstants. This =
allows
> an
> >>>>
> >>>>> Event
> >>>>
> >>>>>>> Handler to indicate that it is willing to relax the event =
ordering
> >>>>
> >>>>>>> requirements so that Event Admin can optimize delivery.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String EVENT_DELIVERY =3D "event.delivery"
> >>>>
> >>>>>>> Service Registration property specifying the delivery =
qualities
> >>>> requested
> >>>>
> >>>>>>> by an Event Handler service.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Event handlers MAY be registered with this property. Each =
value of
> >>>> this
> >>>>
> >>>>>>> property is a string specifying a delivery quality for the =
Event
> >>>> handler.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> The value of this property must be of type String, String[], =
or
> >>>>
> >>>>>>> Collection<String>.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        DELIVERY_ASYNC_ORDERED
> >>>>
> >>>>>>>        DELIVERY_ASYNC_UNORDERED
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String DELIVERY_ASYNC_ORDERED =3D "async.ordered"
> >>>>
> >>>>>>> Event Handler delivery quality value specifying the Event =
Handler
> >>>> requires
> >>>>
> >>>>>>> asynchronously delivered events be delivered in order. Ordered
> >>>> delivery
> >>>>
> >>>>> is
> >>>>
> >>>>>>> the default for asynchronously delivered events.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This delivery quality value is mutually exclusive with
> >>>>
> >>>>>>> DELIVERY_ASYNC_UNORDERED. However, if both this value and
> >>>>
> >>>>>>> DELIVERY_ASYNC_UNORDERED are specifed for an event handler,
> this
> >>>>
> >>>>> value
> >>>>
> >>>>>>> takes precedence.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        EVENT_DELIVERY
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> String DELIVERY_ASYNC_UNORDERED =3D "async.unordered"
> >>>>
> >>>>>>> Event Handler delivery quality value specifying the Event =
Handler
> does
> >>>>
> >>>>> not
> >>>>
> >>>>>>> require asynchronously delivered events be delivered in order.
This
> >>>> may
> >>>>
> >>>>>>> allow an Event Admin implementation to optimize asynchronous
> event
> >>>>
> >>>>>>> delivery by relaxing ordering requirements.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> This delivery quality value is mutually exclusive with
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED. However, if both this value and
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED are specifed for an event handler,
> >>>>
> >>>>>>> DELIVERY_ASYNC_ORDERED takes precedence.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Since:
> >>>>
> >>>>>>>        1.3
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> See Also:
> >>>>
> >>>>>>>        EVENT_DELIVERY
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> --snip--
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> Also see
> >>>>
> >>>>>>> http://stackoverflow.com/questions/7018762/when-to-use-osgi-
> >>>>
> >>>>> eventadmin-and-when-not
> >>>>
> >>>>>>> for BJ's official answer.
> >>>>
> >>>>>>>
> >>>>
> >>>>>>> hope this helps
> >>>>
> >>>>>>> Holger
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
---------------------------------------------------------------------
> >>>>
> >>>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
> >>>>
> >>>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>> --
> >>>>
> >>>>>> View this message in context: http://old.nabble.com/EventAdmin-
> >>>>
> >>>>> Asynchronous-Parallel-EventHandler-tp33158446p33162303.html
> >>>>
> >>>>>> Sent from the Apache Felix - Users mailing list archive at
> Nabble.com.
> >>>>
> >>>>>>
> >>>>
> >>>>>>
> >>>>
> >>>>>>
---------------------------------------------------------------------
> >>>>
> >>>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>>>>
> >>>>
> >>>>>
> >>>>
> >>>>>
> >>>>
> >>>>>
> >>>>
> >>>>> --
> >>>>
> >>>>> Carsten Ziegeler
> >>>>
> >>>>> cziegeler@apache.org
> >>>>
> >>>>>
> >>>>
> >>>>>
---------------------------------------------------------------------
> >>>>
> >>>>> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >>>>
> >>>>> For additional commands, e-mail: users-help@felix.apache.org
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Carsten Ziegeler
> >>> cziegeler@apache.org
> >>
> >>
> >> =
---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> >> For additional commands, e-mail: users-help@felix.apache.org
> >>
> >
> >
> >
> > --
> > Carsten Ziegeler
> > cziegeler@apache.org
> >
> > =
---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> > For additional commands, e-mail: users-help@felix.apache.org
> >
> >
> >
> >
>=20
>=20
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
> For additional commands, e-mail: users-help@felix.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@felix.apache.org
For additional commands, e-mail: users-help@felix.apache.org