jackrabbit-oak-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jukka Zitting <jukka.zitt...@gmail.com>
Subject Re: Oak JCR Observation scalability aspects and concerns
Date Mon, 21 Oct 2013 18:09:20 GMT
Hi,

On Mon, Oct 21, 2013 at 12:03 PM, Carsten Ziegeler <cziegeler@apache.org> wrote:
> This Sling listener is providing higher level application support as it
> creates resource events out of jcr observation events. A lot of code in
> Sling now relies on this functionality and even more higher level code
> based on Sling does so. Usually application listeners register for "/" and
> then filter out to when they handle the event. That's a single string
> operation per handler.
> So imho we should find a solition where this is as fast as possible.

There is ultimately nothing we can do on the repository level to fix this.

Consider a situation where N cluster nodes are each doing just 100
updates per second, changing 10 items per each update, then the Sling
listener would need to be able to process N events per *millisecond*
to keep up. That should be no problem for small values of N and
moderate write loads, but when you consider larger clusters (N > 10,
possibly > 100) and higher write loads (Oak can easily do 1k such
updates per second, and the CreateManyChildNodes benchmark currently
shows TarMK writing over 20k items per second) we reach loads where
the listener would have to process each event in less than a
microsecond, which quickly becomes challenging on modern hardware even
when the handler in most cases just does the mentioned single string
comparison.

The scalability bottleneck here isn't the repository, it's the
non-distributed listener that needs to look at every change across the
cluster.

> Now, if we think about a clustered installation, speaking about the Sling
> application, usually the observation events are not needed on each instance
> in the cluster as the same code is running on all instances. So we could
> think about in this direction as well and find out if delivering the events
> only on the originating instance is working.

Right. This would be a much more scalable approach, and would also
play well with the limitation that we extra user data is not available
for observation events originating from external cluster nodes.

> However, there are many use cases where a local cache is used on
> each instance which would make such an approach useless.

Oak has a variety of solutions that could be used to make such code
more scalable (ordered by increasing effectiveness / decreasing
implementation-independence):

1) Oak is in often faster than Jackrabbit 2.x especially for
concurrent access, so there might be no need for caching at all in at
least some of the cases.

2) Custom index definitions can be used to turn expensive
traversal/filtering accesses to efficient queries.

3) The Observer mechanism allows a listener to look at repository
changes in variable granularity and frequency depending on application
needs and current repository load. Thus an Oak Observer can
potentially process orders of magnitude more changes than a JCR event
listener that needs to look at each individual changed item.

4) Commit hooks can be used to maintain an in-repository cache in a
fully distributed manner.

BR,

Jukka Zitting

Mime
View raw message