jackrabbit-oak-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carsten Ziegeler <cziege...@apache.org>
Subject Re: Oak JCR Observation scalability aspects and concerns
Date Mon, 21 Oct 2013 16:03:35 GMT
This Sling listener is providing higher level application support as it
creates resource events out of jcr observation events. A lot of code in
Sling now relies on this functionality and even more higher level code
based on Sling does so. Usually application listeners register for "/" and
then filter out to when they handle the event. That's a single string
operation per handler.
So imho we should find a solition where this is as fast as possible.

Right now, the Sling listener does two things, it compacts the jcr
observation events and then reads each created/modified node to find out
the resource (super) type (with one additional special case).
Obviously, this isn't optimal, especially the read of every changed node -
but as there is no other way to get the values of the two properties, this
is how it is :)
I'm pretty sure that this can be optimized.

Of course, this should be independent of analysing the other parts as
discussed in thread.

Now, if we think about a clustered installation, speaking about the Sling
application, usually the observation events are not needed on each instance
in the cluster as the same code is running on all instances. So we could
think about in this direction as well and find out if delivering the events
only on the originating instance is working. However, there are many use
cases where a local cache is used on each instance which would make such an
approach useless.

Carsten



2013/10/21 Jukka Zitting <jukka.zitting@gmail.com>

> Hi,
>
> On Mon, Oct 21, 2013 at 9:47 AM, Bertrand Delacretaz
> <bdelacretaz@apache.org> wrote:
> > On Mon, Oct 21, 2013 at 3:17 PM, Jukka Zitting <jukka.zitting@gmail.com>
> wrote:
> >> ...Instead of an repository problem (like diffing, event creation,
> etc.),
> >> this analysis tells me that the bottleneck here is the application
> >> that tries to listen to so many events...
> >
> > FWIW, by default Sling does have a listener that catches everything to
> > rebroadcast JCR observation events as OSGi events.
> >
> > See JcrResourceListener [1] which is created by
> > JcrResourceProviderFactory [2] which by default causes it to listen
> > for all node/property added/removed/changed events on /.
> >
> > That's difficult to modify while staying backwards compatible, as we
> > have no way of knowing which events are actually used.
> >
> > I don't have a suggestion at this point, just wanted to make sure
> > you're aware of this.
>
> Right, it just means that a deployment with such an observer will have
> a built-in scalability limit as at some point the listener will no
> longer be able to keep up with all concurrent writes across a large
> and busy cluster.
>
> For now I'd document this limitation and possibly deprecate the
> JcrResourceListener functionality. A deployment can turn the
> functionality off once it reaches the scalability limit and has
> identified/fixed all affected code.
>
> BR,
>
> Jukka Zitting
>



-- 
Carsten Ziegeler
cziegeler@apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message