jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Toper" <nto...@gmail.com>
Subject Re: Google Summer of Code project for Jackrabbit
Date Wed, 24 May 2006 11:12:03 GMT
Hi,

Thanks to Jukka, the ASF and Google, I have been selected for the Google
Summer of Code. Jukka Zitting is mentoring this project. The project is on
from now until August 21.

Our goal is to create a backup tool for JackRabbit. Here is the project
initial description:

"Implement a tool for backing up and restoring content in an Apache
Jackrabbit content repository. In addition to the basic content hierarchies,
the tool should be able to efficiently manage binary content, node version
histories, custom node types, and namespace mappings. Incremental or
selective backups would be a nice addition, but not strictly necessary."

I have started working on the design and I am asking for your feedback to
improve it. Please tell me if it suits your needs and what should be
improved, what are your ideas on this subject and anything you would like.


This feedback and design phase should be over in at most two weeks so I can
start coding.



*Project Goals*
We have set up these design goals:


> Backup both data and configuration option.
> Ease of use for sysAdmin to backup and restore content.
> Aim for generic operations: when adding new functionalities to JackRabbit
we should not have to update the backup application code.
> Aim for non blocking operations. The repository should work normally when
being backuped whenever possible.
> Aim for modularity. This would be the first release of the backup tool. It
will evolve for sure.
> Disk space is not an issue for now. (It can be worked out in another
release)
> The tool is specific to JackRabbit as mysqldump is to MySQL.



*Architecture*
The application is composed of those modules:

/A configuration XML file (part of repository.xml?). It would contain
pointers to the resources to backup (content, hierarchies, custom node
types, Lucene index file, repository.xml, and so on) in Xpath form (relative
to the repository.xml file) and "file system path". Whenever possible, we
will gather informations from JackRabbit configuration files.

/ The backup format is still to be defined. We would like to use any
PersistenceManager. This way we can achieve greater flexibility. For
instance backuping to another DB or to the file system. Using a
PersistenceManager as a backup format allows us to migrate seemingly between
PersistenceManager. Each supported PersistenceManager will have its backup
strategy (possibly using external tools). For now we think to support only
the ObjectPersistenceManager or the XMLPersistenceManager (which means
fetching the directory).

/ A GenericConverter class to copy a workspace into another. For the backup,
the destination workspace will be a temporary workspace configured with a
specific PersistenceManader and a File System. This way we will achieve the
backup using a pivot format. This class can have other use for the
JackRabbit project.

/ The backup module would work according to the following pseudo-code:

Backup all repository elements (repository.xml for instance)
List all workspaces
For each workspace
Backup specific data (Lucene index file for instance) in working directory
Copy workspace using GenericConverter to the backup repository

End
Save XML System view file.
Zip all data and move them to a specific folder.
Delete backup repository

/ The Restoration module would work this way:
Unzip backup data in the working directory
Restore repository data
For each workspace:
Restore specific data
Restore content using importXML function.
End
Delete working directory.

*NB *

- We will use to backup the data if possible the existing Jackrabbit or
another installation (depending on load test and your recommandations). We
would backup one workspace at a time in a temporary workspace. After the
data would be backuped, we would destroy and recreate the workspace. Is it
possible in the long run (backup would be performed once a day) or it is not
an advisable strategy?

- All updates and read are isolated through transactions. Do we need to
define a locking strategy? If I am correct, I can read a node even though it
is locked and it is threadsafe. You don't commit an incoherent modification.

- We can ignore transient items for now (the use case we work on is regular
backup; besides the admin can forbid connexion for specific backup)

- Another approach is to use the system view. But how would we handle
JackRabbit extension? What do you think?

*Functionalities*
The backup/restore application will allow at least to configure those
parameters:

>From the Command line
backup only a specific workspace
backup all workspaces

 From the XML configuration file:
Backup Data: one use case would be to create a Jackrabbit installation
correctly configured but with no data in it. We would not want to allow data
in it but still want the configuration parameters.
List of all specific elements to backup (see upper): this way the software
is easily extensible and will follow JackRabbit evolution.

*Other approaches*
Other approaches have been evaluated:

> Working only on the file level is simple to implement. But this solution
would not make it possible to switch between persistence mechanisms. For
example a common use case is that a user has started with some simple
PersistenceManager, but then needs to switch to a more efficient one as the
usage patterns change. It would be a nice addition if the backup tool could
be used to handle such migrations as well as standard backup-restore
situations.

> Working on the JCR API level only. The problem with this approach is that
the JCR API
does not specify any way to import or directly modify version histories or
node types.

*Deliverables*
- Two Java application: backup and restore.
- XML configuration file (one for both)
- Documentation (in the JackRabbit Wiki + Javadoc)
- Maybe some patch to the JackRabbit project (if needed, for instance
importXML might need some rework).

*Evolution (after GSOC)*
- Add a sanity check to the archive just made.
- Add incremental backup (this would mean a hashcode on each node and a test
to know whether or not it had been changed).

URL to the original Google SoC Application:
http://www.deviant-abstraction.net/application-for-google-summer-of-code-2006/


I look forward for your feedback and I am happy to work with all of you on
this tool. I will give my best to make this project a success with your
help.



Nicolas

My blog! http://www.deviant-abstraction.net !!

On 5/24/06, Jukka Zitting <jukka.zitting@gmail.com> wrote:
>
> Hi,
>
> Good news! We've just been granted a Google Summer of Code project for
> creating a backup tool for Jackrabbit. Please welcome Nicolas Toper as
> the student who will be working on the project. I will act as Nicolas'
> mentor.
>
> Nicolas will soon send more details and his initial design ideas for
> the tool. Please comment on his work as you would on any external
> contribution.
>
> See the Google Summer of Code page (http://code.google.com/soc/) for
> more details on the program.
>
> BR,
>
> Jukka Zitting
>
> --
> Yukatan - http://yukatan.fi/ - info@yukatan.fi
> Software craftsmanship, JCR consulting, and Java development
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message