jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Klimetschek <aklim...@day.com>
Subject Re: [jr3] Clustering: Scalable Writes / Asynchronous Change Merging
Date Thu, 21 Oct 2010 14:57:37 GMT
On Wed, Oct 20, 2010 at 11:22, Thomas Müller <thomas.mueller@day.com> wrote:
> Let's discuss partitioning / sharding in another thread. Asynchronous
> change merging is not about how to manage huge repositories (for that
> you need partitioning / sharding),

Having different consistency expectations for certain parts in order
to make certain writes faster is not necessarily about huge
repositories.

> But I don't want that "the network is the new disk" (in terms of
> bottleneck, in terms of performance problem). Networking delay may be
> the bottleneck even if cluster nodes are in the same room, specially
> when you keep the whole repository in memory, or use SSDs.

Network delay in a proper setup is faster than the delay of a disk
drive nowadays (leaving out the other end of the network that has to
store the data it receives).

For some numbers, see
http://www.tele-task.de/archive/video/flash/11097/ slide starting at
about 3:21 (until ~7:00).

This underlies the importance of leveraging in-memory storage, btw, as
Jukka also pointed out.

> Asynchronous operation is bad for reservation systems, banking
> applications, or if you can't guarantee sticky sessions. Here you need
> synchronous operations or at least locking. If you want to support
> both cases in the same repository, you could use virtual repositories
> (which are also good for partitioning / sharding).

We could use virtual repositories for that, but that should be more
easily configurable then and make sure it is performant. Still, it
would need to be configurable.

> I think the persistence API should be synchronous
> as it is now.

What is then the reason for asynchronous change merging, if not for
performance? AFAIU, the synchronous operation of the (cloud)
persistence layer makes it quite slow compared to a pure local storage
(especially if that one is in-memory).

Regards,
Alex

-- 
Alexander Klimetschek
alexander.klimetschek@day.com

Mime
View raw message