jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jukka Zitting <jukka.zitt...@gmail.com>
Subject Re: [jr3 microkernel] Write skew
Date Thu, 01 Dec 2011 15:34:22 GMT

On Thu, Dec 1, 2011 at 3:49 PM, Thomas Mueller <mueller@adobe.com> wrote:
> But I don't think we should try to increase concurrency of
> write operations within the *same* repository [...]

That's a valid solution, but it means that we explicitly drop the idea
of being able to scale concurrent writes linearly by adding new
cluster nodes. See also http://markmail.org/message/nuxclhqsbihanbjg
for the original discussion about this.

> - in a cluster, writes do not scale as writing is basically synchronized

This I think is the key issue here. Do we want to break the
synchronization requirement and thus eliminate this bottleneck
entirely, or are we happy with more localized performance improvements
for the concurrent write case?

Ultimately the question boils down to whether we want to support
massively parallel systems where the number of concurrent writers
(measured as writes happening within the communication latency across
the entire cluster) starts at thousands and can ultimately scale go up
millions. Such systems are practically impossible to implement with
any non-zero level of global synchronization, which is why I think
this is such a crucial design decision to be made.

I'm fine if we explicitly decide not to target such use cases. Then we
can avoid having to worry about the complexities of an eventual
consistency model. With good optimization it's still possible to
achieve pretty impressive concurrent performance even with a
synchronized cluster.


Jukka Zitting

View raw message