ace-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcel Offermans <>
Subject Re: Scalable and highly available data store
Date Thu, 29 May 2014 23:42:40 GMT
Hello Wayne,

On 30 May 2014, at 0:12 am, Wayne Nelson <> wrote:

> The org.apache.ace.repository.Repository interface is an abstraction over the persistent
data store. However, it doesn't appear to support arbitrary queries. Instead, you must call
checkout() and unmarshal an XML document from the returned InputStream. This means that the
entire data store must be able to reside in memory.

Yes and no.

ACE consists of a "server" and a "client" part (which can be unified into one, but don't have
to be).

The ACE server is where the targets connect, and when a target checks for updates, it will
use a SAX parser to just get the parts it needs for that request.

The ACE client is where you manipulate the configuration. There the whole repository is indeed
unmarshalled into memory, so the amount of targets is limited by memory here.

> This doesn't seem like it will scale well to hundreds of thousands of targets in a single
data store.

Probably not, no, but you can partition the targets into separate stores (we have a mechanism
that is similar to multitenancy for that, you can create different instances of the stores
for specific "customers").

> The commit() method uses optimistic locking, which means that only one thread can update
the database at a time.

That is correct, the current client has no feature to "merge" results that have been committed
by a different client, so all a client can then do is a new checkout and then apply the changes
it wanted to do again.

> This seems like we are limited to a single point of failure with a single server serving
up all requests from all agents.

When agents talk to the server, there is no locking at all. Locking is only necessary when
updating the configuration, so for example when you upload new bundles, or add new targets/agents.
Also, the repositories can be replicated, so agents can connect to any server.

Greetings, Marcel

View raw message