jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Dürig <mdue...@apache.org>
Subject Re: [jr3 microkernel] Write skew
Date Thu, 01 Dec 2011 17:54:46 GMT


On 1.12.11 15:34, Jukka Zitting wrote:
>> But I don't think we should try to increase concurrency of
>> write operations within the *same* repository [...]
>
> That's a valid solution, but it means that we explicitly drop the idea
> of being able to scale concurrent writes linearly by adding new
> cluster nodes. See also http://markmail.org/message/nuxclhqsbihanbjg
> for the original discussion about this.
>
>> - in a cluster, writes do not scale as writing is basically synchronized
>
> This I think is the key issue here. Do we want to break the
> synchronization requirement and thus eliminate this bottleneck
> entirely, or are we happy with more localized performance improvements
> for the concurrent write case?
>
> Ultimately the question boils down to whether we want to support
> massively parallel systems where the number of concurrent writers
> (measured as writes happening within the communication latency across
> the entire cluster) starts at thousands and can ultimately scale go up
> millions. Such systems are practically impossible to implement with
> any non-zero level of global synchronization, which is why I think
> this is such a crucial design decision to be made.
>
> I'm fine if we explicitly decide not to target such use cases. Then we
> can avoid having to worry about the complexities of an eventual
> consistency model. With good optimization it's still possible to
> achieve pretty impressive concurrent performance even with a
> synchronized cluster.

One of the main goals of the Microkernel architecture was to get true 
write scalability. If we go down that route, we have to accept that we 
are trading this for consistency.

Michael


Mime
View raw message