jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miro Walker" <miro.wal...@cognifide.com>
Subject RE: Google Summer of Code project for Jackrabbit
Date Wed, 24 May 2006 17:03:58 GMT
In case it's helpful, I thought I'd give you an idea of how we are
currently handling backup (and related use cases, such as failover and
disaster recovery) in our application.

We have set the system up to use a single database instance to store all
mutable (transactional) data in the system (except for search indexes).
This is done by using a database PM and the DBFileSystem. The database
to which the system writes is then used to handle all backup, restore,
etc. This allows us to rely on the underlying database capability (in
our case MS SQL Server) to handle backup through transfer of transaction
logs. Because the DB handles the work, backup is completely invisible to
the application and respects transactions properly. 

I think the key feature here is that databases are designed to persist
data via transaction logs, so a backup operation can be initiated and
will write the data as it was at that point in time, without having to
worry about data that is written during the backup making the system
inconsistent. Because jackrabbit doesn't use such an approach it may
well be unavoidable to lock all data before initiating a backup.

Disaster Recovery is handled by hosting the database on a SAN that is
replicated between datacentres, allowing immediate DR failover without
data loss.

Application failover within a single datacentre is handled by using the
shared database, a shared file system area (to hold the repository lock
file) and separate search indexes per application instance. Some custom
code and an external load balancer box then handle the activation and
passivation of application instances.

There are a couple of downsides to this approach, however:

1. The current design of jackrabbit DBFileSystem stores database
connection strings within the database itself (as these are embedded in
workspace.xml files). This means that transferring backups from one
system to another requires running a separate app before the main
application can be used to extract the workspace.xml files from the
DBFileSystem, modify the DB connection strings and then write them back.
Either that or you use standard username/password and tricks with local
dns config to ensure that the database can be accessed using the same
strings regardless of environment.

2. Search indexes must be completely rebuilt after restore - in our
case, with many (150+) workspaces and large volumes of content, this can
take hours. We get around this by not re-indexing all workspaces at the
same time, but it's still annoying.

Cheers,

Miro

Mime
View raw message