Return-Path: X-Original-To: apmail-jackrabbit-commits-archive@www.apache.org Delivered-To: apmail-jackrabbit-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DCBDDE371 for ; Mon, 31 Dec 2012 12:41:32 +0000 (UTC) Received: (qmail 10331 invoked by uid 500); 31 Dec 2012 12:41:32 -0000 Delivered-To: apmail-jackrabbit-commits-archive@jackrabbit.apache.org Received: (qmail 10283 invoked by uid 500); 31 Dec 2012 12:41:32 -0000 Mailing-List: contact commits-help@jackrabbit.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@jackrabbit.apache.org Delivered-To: mailing list commits@jackrabbit.apache.org Received: (qmail 10265 invoked by uid 99); 31 Dec 2012 12:41:31 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Dec 2012 12:41:31 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 31 Dec 2012 12:41:27 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id DE9E32388C22 for ; Mon, 31 Dec 2012 12:41:04 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r844404 [2/17] - in /websites/staging/jackrabbit/trunk/content: ./ JCR/ JCR/a-simple-ocm-project-with-maven-eclipse.data/ JCR/concurrency-control.data/ JCR/deployment-models.data/ JCR/how-jackrabbit-works.data/ JCR/how-to-map-associations-b... Date: Mon, 31 Dec 2012 12:40:59 -0000 To: commits@jackrabbit.apache.org From: buildbot@apache.org X-Mailer: svnmailer-1.0.8-patched Message-Id: <20121231124104.DE9E32388C22@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Added: websites/staging/jackrabbit/trunk/content/JCR/building-jackrabbit.html ============================================================================== --- websites/staging/jackrabbit/trunk/content/JCR/building-jackrabbit.html (added) +++ websites/staging/jackrabbit/trunk/content/JCR/building-jackrabbit.html Mon Dec 31 12:40:56 2012 @@ -0,0 +1,143 @@ + + + + + + + Building Jackrabbit + + + +
+ +
+ +
+

Building Jackrabbit

+

The easiest way to use Jackrabbit is to download a binary release, but if you want to access the latest development version, you need to get the Jackrabbit sources and build them using the Maven 2 build environment.

+ +

The first step in building Jackrabbit is to check out the Jackrabbit sources from the Subversion source repository at http://svn.apache.org/repos/asf/jackrabbit/. The source tree is divided in standard parts: trunk, branches, tags and commons (more info about commons). The latest development version is found within trunk, while the other parts are used to keep track of the source code of the Jackrabbit releases.

+ +

The trunk contains the top-level build environment and a number of component projects within subdirectories. See the README.txt files within each subdirectory for a brief description of the component project. There is also a sandbox directory with miscellaneous contributions that are not yet a part of the official Jackrabbit releases.

+ + +

Checking out the sources with Subversion

+ +

You need a Subversion client to access the Jackrabbit source repository. Take a look at the Subversion client list unless you already have a one installed. Once you have the Subversion client installed you can checkout the main Jackrabbit source tree with the following command or its equivalent in the client you are using:

+
+
+svn checkout http://svn.apache.org/repos/asf/jackrabbit/trunk jackrabbit
+
+
+

The above checkout will create a subdirectory named jackrabbit that contains the latest Jackrabbit sources. See the Subversion book or the documentation of your Subversion client for more information on how to manage your source tree and keep it up to date with latest development.

+ + +

Building the sources with Maven

+ +

Jackrabbit uses Maven 2 as the build system and the component sources are mostly organized according to the Maven Standard Directory Layout.
+The standard build environment is Maven 2 with the Java Development Kit (JDK) 1.5 (but the build is known to work also with Java 6).

+ +
Jackrabbit 1.x
The standard build environment for 1.x versions is Maven 2 with the Java Development Kit (JDK) 1.4 (but the build is known to work also with Java 5 and 6).
+ +

See the Running Maven page and the related documentation on the Maven web site for instructions on how to use Maven. You may also want to check for Maven 2 integration with your favourite Integrated Development Environment (IDE).

+ +

There are Maven project descriptors (POMs) within both the top level jackrabbit directory you checked out above and all the jackrabbit-something component subdirectories. The easiest way to build Jackrabbit is to use the "multimodule" setup within the top level directory:

+
+
+$ cd /path/to/jackrabbit; mvn install
+
+
+

This will build and package all the component projects and place the resulting artifacts within your local Maven 2 repository. You can also find the artifacts within the created target subdirectories of the component projects.

+
+
+
+ +
+ + + + + Added: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/arch.jpg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/arch.jpg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/arch.jpg.jpeg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/arch.jpg.jpeg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/deadlock.jpg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/deadlock.jpg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/deadlock.jpg.jpeg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/deadlock.jpg.jpeg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/lock.jpg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/lock.jpg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/lock.jpg.jpeg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.data/lock.jpg.jpeg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.html ============================================================================== --- websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.html (added) +++ websites/staging/jackrabbit/trunk/content/JCR/concurrency-control.html Mon Dec 31 12:40:56 2012 @@ -0,0 +1,403 @@ + + + + + + + Concurrency control + + + +
+ +
+ +
+

Concurrency control

+

The internal concurrency model in Apache Jackrabbit is fairly complex and a number of deadlock issues have been reported and fixed over the Jackrabbit 1.x release cycle. This document is the result of a design and code review targeted at proactively preventing other similar issues.

+ +

This document is about the internal concurrency and synchronization model in Jackrabbit, not about the JCR locking feature. Note that the review that led to this document targeted concurrency control at an architectural level and did not focus much on issues like thread-safety of individual classes or components.

+ +

This review is based on Jackrabbit version 1.5 in default configuration.

+ +

Architectural background

+ +

In terms of concurrency control, the Jackrabbit architecture can roughly be divided to five main layers:

+ +
    +
  1. Cluster
  2. +
  3. Repository
  4. +
  5. Workspace
  6. +
  7. Session
  8. +
  9. Transaction
  10. +
+ + +

The clustering layer takes care of synchronizing changes across one or more cluster nodes that are each treated as individual repositories that happen to share their content. Concurrency control across different cluster nodes is achieved using a single write lock that a cluster node is required to acquire before it can commit any changes to the shared state. On the other hand all cluster nodes can read the shared content in parallel with no explicit synchronization. Note that since the cluster nodes only share a single lock, a deadlock can not occur between the locks in one node and the ones in another. A single deadlocked node can still potentially block writes to the entire cluster, but the clustering feature can not add any new deadlock scenarios if each node would be deadlock-free by itself.

+ +

The repository layer takes care of all global repository state like the node type registry and the version storage. Instead of a single global repository lock, all the repository-wide components have their own synchronization mechanisms. The most notable component from a concurrency control point of view is the version storage, that actually contains two locking mechanisms; one in VersionManagerImpl for high level versioning operations and one in the underlying SharedItemStateManager for controlling access to the underlying persistence mechanism.

+ +

A repository consists of one or more workspaces that contain the normal content trees of the repository. Each workspace consists of a few components like the persistence mechanism and the search index. The persistence mechanism is built from a SharedItemStateManager that controls all item operations and a PersistenceManager that persists items in permanent storage. Most persistence managers use Java synchronization or some other locking mechanism for concurrency control, but since they typically don't interact much with other parts of the repository they are not that critical from a global concurrency perspective. On the other hand, the SharedItemStateManager that uses a read-write lock is a key element especially given the way it interacts with the repository-wide version store. Note that since Jackrabbit 1.4 it has been possible to configure locking strategy of the SharedItemStateManager to use a more fine-grained set of locks to allow concurrent write access to differe nt parts of the content tree. This review focuses on the default case of having just a single SharedItemStateManager lock, but from a locking perspective the more fine-grained case is roughly equivalent to having more workspaces and thus the results of this review should still apply.

+ +

Each workspace can be accessed by zero or more JCR sessions. Each session contains a transient space that keeps track of all unsaved changes in that session. Since the transient space is local to a session and since a session should only be accessed by one thread at a time, there are few concurrency concerns associated with the use of sessions. However, note that the thread-safety requirements of sessions are in many cases not explicitly enforced by Jackrabbit, so a client that intentionally or accidentally uses a single session from multiple concurrent threads may well end up corrupting the internal state of the session.

+ +

Transactions are handled in Jackrabbit by wrapping all item operations (saved transient changes and direct workspace updates, as well as versioning and locking operations) into a sort of a larger transient space that gets persisted only when the transaction is committed. There is no "transaction lock" in Jackrabbit, but transaction support still fundamentally changes Jackrabbit concurrency control as it basically replaces all write operations (and related locking) with the larger commit operation. This transaction mode is only activated when a session is within the context of an XA transaction.

+ +

Main synchronization mechanisms

+ +

The main synchronization mechanisms in Jackrabbit are the read-write locks in the SharedItemStateManager and VersionManagerImpl classes. Other components also have concurrency control features, for example the LockManagerImpl class (used for handling JCR locks) uses a reentrant lock and the NodeTypeRegistry class relies on Java synchronization. This review focuses on just the two main components as those are by far the most actively used and the ones that could potentially block all access to repository content in case of a deadlock. The three main locks to be concerned about are:

+ +
    +
  • "Workspace lock", the read-write lock of the per-workspace SharedItemStateManager
  • +
  • "Versioning lock", the read-write lock of the repository-wide VersionManagerImpl
  • +
  • "Version store lock", the read-write lock of the SharedItemStateManager associated with the version manager
  • +
+ + +

Each of these locks can be locked exclusively for write access or inclusively for read access. In other words, any number of concurrent readers can keep the lock, but any single writer will block out all other readers and writers.

+ +

As noted in the section above, the workspace locks may also be collections of more finely grained locks, but this review concentrates on the default case. Note also that each workspace has it's own lock, so even if one workspace is exclusively locked, other workspaces can still be accessed.

+ +

Conditions for deadlocks

+ +

A deadlock can only occur if the holder of one lock tries to acquire another lock and there is another thread (or a series of other threads) that tries to do the reverse. This situation can only arise if a) locks are acquired in a nested sequence, b) different threads can acquire the nested locks in a different order, and c) at least two exclusive locks are being acquired.

+ +

Most operations in Jackrabbit avoid deadlocks in one of the following three ways:

+ +
    +
  • Only a single lock is held at a time, breaking condition a. This case covers most of the code doing sanity checks and other preparing work associated with many operations.
  • +
  • In case of nested locks, the code guarded by the inner lock never tries to acquire another lock, breaking condition b. This case covers for example the numerous calls to the underlying persistence managers that typically have their own synchronization mechanism but never call out to other Jackrabbit components except perhaps the namespace registry that also satisfies this condition.
  • +
  • None of the nested locks are exclusive. This covers all read operations in Jackrabbit, so a deadlock can never occur if all clients only read from the repository.
  • +
+ + +

The potentially troublesome cases are two or more concurrent write operations with nested locks, or a write operation with two nested exclusive locks running concurrently with read operations with nested locks. See below for the results of the code review that tried to identify and clear such cases. The acquired write locks are marked in bold to make it easy to spot potential problems.

+ +

Code review

+ +

This section contains the results of a code review whose purpose was to identify the order and nesting of the locks acquired by many common operations in Jackrabbit. The results of the review were compared to the above conditions for deadlock.

+ +

Note that the descriptions of the write operations below assume non-transactional context. See the last subsection for the behaviour in transactional environments.

+ +

Normal read access

+ +

Read access to the workspace typically only requires a read lock on the SharedItemStateManager of that workspace, but since the version store is mapped to the virtual /jcr:system/jcr:versionStorage inside the repository, there are cases where also the read lock of the version store needs to be acquired.

+ +
    +
  1. Workspace read lock, for reading normal node content +
      +
    1. Version store read lock, for reading version content
    2. +
    +
  2. +
+ + +

This nested lock is potentially unsafe in a transactional context, see the subsection on transaction commit below for more details.

+ +

Versioning read access

+ +

Some version accesses are handled directly through the version manager instead of looking through the /jcr:system/jcr:versionStorage tree. Such accessed are guarded with the VersionManagerImpl read lock.

+ +
    +
  1. Versioning read lock, for accessing version information +
      +
    1. Version store read lock, for reading version information
    2. +
    +
  2. +
+ + +

The nested lock here is safe as the version store lock never covers code that tries to acquire the versioning lock.

+ +

Transient changes

+ +

All transient changes like those created by Node.addNode() or Session.move() are stored in the session-local transient space without needing any synchronization except for the read locks used for accessing the underlying workspace state. A write lock is only needed when the accumulated changes are being persisted using the save() call described below.

+ +

Save

+ +

The ItemImpl.save() method (that SessionImpl.save() also calls) collects all current transient changes to a single change log that is then persisted as an atomic update. Any new versionable nodes will cause empty version histories to be created in the version store. Note that ItemImpl.save() is synchronized on the current session, enforcing the rule that no two threads should be concurrently using the same session.

+ +
    +
  1. Workspace read lock, for sanity checks and other preliminary work
  2. +
  3. Multiple non-overlapping instances of (only when creating new version histories) +
      +
    1. Workspace read lock, for checking the current state of the nodes being modified
    2. +
    3. Version store read lock, for checking whether a version history already exists
    4. +
    5. Versioning write lock, for creating a new version history +
        +
      1. Version store write lock, for persisting the version history
      2. +
      +
    6. +
    +
  4. +
  5. Workspace write lock, for persisting the changes +
      +
    1. Version store read lock, for checking references
    2. +
    3. Version store write lock, for persisting updated back-references
    4. +
    +
  6. +
+ + +

Many of the other write operations below call ItemImpl.save() internally to persist changes in the current workspace. However, in the descriptions I've only included the last "Workspace write lock" branch (with the "Version store write lock" excluded if it's clear that no back-references need to be updated) as the operations are guaranteed to never contain cases where new version histories would need to be created.

+ +

Here we have three cases of nested locks involving one or more exclusive locks:

+ +
    +
  • Versioning write lock -> Version store write lock
  • +
  • Workspace write lock -> Version store read lock
  • +
  • Workspace write lock -> Version store write lock
  • +
+ + +

All these nested locks are safe in non-transactional context since the version store lock never covers code that tries to acquire one of the other locks. The same is true for the first case also in transactional context, but see the transaction commit subsection below for a discussion of how the other two cases are different with transactions.

+ +

Merge and update

+ +

The Node.merge() and Node.update() methods both call NodeImpl.internalMerge() that acquires a new session on the source workspace and copies relevant content to the current workspace.

+ +
    +
  1. Multiple non-overlapping instances of +
      +
    1. Source workspace read lock, for copying content to the current workspace
    2. +
    3. Current workspace read lock, for comparing current status with the one being merged
    4. +
    +
  2. +
  3. Current workspace write lock, for persisting the changes +
      +
    1. Version store read lock, for checking references
    2. +
    3. Version store write lock, for persisting updated back-references
    4. +
    +
  4. +
+ + +

The nested locks above are discussed in the section on the save operation.

+ +

Copy, clone and move

+ +

The various copy(), clone() and move() methods in WorkspaceImpl use the similarly called methods in BatchedItemOperations to perform batch operations within a single workspace or across two workspaces. From a synchronization perspective these operations are much like the merge and update operations above, the difference is mostly that the source workspace may be the same as the current workspace.

+ +
    +
  1. Multiple non-overlapping instances of +
      +
    1. Source workspace read lock, for copying content to the current workspace
    2. +
    3. Current workspace read lock, for comparing current status with the one being copied
    4. +
    +
  2. +
  3. Current workspace write lock, for persisting the changes +
      +
    1. Version store read lock, for checking references
    2. +
    3. Version store write lock, for persisting updated back-references
    4. +
    +
  4. +
+ + +

The nested locks above are discussed in the section on the save operation.

+ +

Checkin

+ +

The NodeImpl.checkin() method first creates a new version of the node in the shared version store and then updates the appropriate mix:versionable properties of the node.

+ +
    +
  1. Workspace read lock, for sanity checks and other preliminary work
  2. +
  3. Versioning write lock, for creating the new version in the version store +
      +
    1. Workspace read lock, for copying content to the new version
    2. +
    3. Version store write lock, for persisting the new version
    4. +
    +
  4. +
  5. Versioning read lock, for accessing the newly created version +
      +
    1. Version store read lock, for reading the new version
    2. +
    +
  6. +
  7. Workspace write lock, for updating the node with references to the new version +
      +
    1. Version store read lock, for checking references
    2. +
    3. Version store write lock, for persisting updated back-references
    4. +
    +
  8. +
+ + +

The overlapping lock region above is not troublesome as there are no cases where a versioning lock is acquired within the scope of a workspace lock. Note that there previously were such cases, but this code review shows that all of them have since been solved.

+ +

The nested locks above are discussed in the sections on versioning read access and the save operation.

+ +

Checkout

+ +

The NodeImpl.checkout() method simply does some sanity checks and updates the versioning metadata of the node to reflect the changed state. No access to the shared version store is needed.

+ +
    +
  1. Workspace read lock, for sanity checks
  2. +
  3. Workspace write lock, for updating the node to reflect the checked out state +
      +
    1. Version store read lock, for checking references
    2. +
    +
  4. +
+ + +

The nested lock above is discussed in the section on the save operation.

+ +

Restore

+ +

The various Node.restore() and Workspace.restore() methods all end up calling NodeImpl.internalRestore() that copies contents of the selected version back to the workspace. Finally the changes are persisted with a ItempImpl.save() call.

+ +
    +
  1. Multiple non-overlapping instances of: +
      +
    1. Versioning read lock, for copying content back to the workspace
    2. +
    3. Workspace read lock, for comparing the current state with the version being restored
    4. +
    +
  2. +
  3. Workspace write lock, for persisting the changes +
      +
    1. Version store read lock, for checking references
    2. +
    3. Version store write lock, for persisting updated back-references
    4. +
    +
  4. +
+ + +

The nested locks above are discussed in the section on the save operation.

+ +

Transaction commit

+ +

As discussed in the architecture section above, a transaction context overrides all the other write operations in favor of the two-phase commit driven by the transaction manager. The Jackrabbit part of a potentially distributed transaction is coordinated by the XASessionImpl class that causes the following locking behavior:

+ +
    +
  1. Versioning write lock, for the entire commit +
      +
    1. Version store write lock, for persisting modified version histories +
        +
      1. Workspace write lock, for persisting modified content
      2. +
      +
    2. +
    +
  2. +
+ + +

The curious ordering of the locks is caused by the way the prepare and commit parts of the different transaction components are nested. This nesting of the workspace lock within the version store lock is a bit troublesome in comparison with the nesting in read operations and non-transactional writes where the order of the locks is reverse. The nesting order here can not be easily changed as any new versions and version histories need to be persisted before workspace content that refers to them. Possible solutions could be either to disable or redesign the reference checks done in a transactional context, or to relax transaction semantics by persisting the version history changes already in the prepare phase in which case the version store lock wouldn't need to cover the workspace lock. However, even before this issue is fixed, the impact is quite limited and can easily be worked around by typical clients.

+ +

In read operations the version store read lock is only acquired after the workspace lock if reading content in /jcr:system/jcr:versionStorage. Clients that never looks at the /jcr:system/jcr:versionStorage tree and uses the JCR API methods like getVersionHistory() to access version information will not trigger the potential deadlock case.

+ +

Write operations can only cause a deadlock when both transactional and non-transactional writes are performed concurrently against the same repository. A repository that is consistently accessed either transactionally or non-transactionally will not trigger this deadlock. Note that this restriction is workspace-specific, i.e. one workspace can safely be written to transactionally even if another workspace is concurrently written to non-transactionally.

+ +

Summary and future work

+ +

This review shows that while the internal locking behaviour in Jackrabbit is still far from simple, there aren't any major deadlock scenarios remaining. The two issues identified in the review can be easily avoided by following these two rules:

+ +
    +
  • Use the JCR versioning API instead of the /jcr:system/jcr:versionStorage tree to access version information
  • +
  • Don't mix concurrent transactional and non-transactional writes to a single workspace
  • +
+ + +

The transaction commit subsection above outlines some possible solutions to make even these workarounds unnecessary.

+ +

The following other potential improvements were identified during the code review:

+ +
    +
  • Storing the version history back-references in the workspaces that contain the references would simplify a lot of code and remove a major source of interaction between the workspace and version store when updating content. The downside of this change is that removing versions and version histories would be much more difficult as all workspaces would need to be checked for potential references.
  • +
  • The current design contains lots of cases where read locks are acquired and released multiple times in sequence. This is often caused by the need to check the transient space when reading something from the repository. It might be useful to extend the workspace read lock to cover also all the transient spaces even when the transient spaces would still be session-specific.
  • +
  • Adopting a single global repository lock for all per-repository components would simplify lots of code at the expense of some performance.
  • +
+ + +
+
+
+ +
+ + + + + Added: websites/staging/jackrabbit/trunk/content/JCR/continuous-integration.html ============================================================================== --- websites/staging/jackrabbit/trunk/content/JCR/continuous-integration.html (added) +++ websites/staging/jackrabbit/trunk/content/JCR/continuous-integration.html Mon Dec 31 12:40:56 2012 @@ -0,0 +1,116 @@ + + + + + + + Continuous Integration + + + +
+ +
+ +
+

Continuous Integration

+

Apache Jackrabbit uses Hudson for continuous integration builds. The Jackrabbit builds are a part of the Hudson zone in the Apache infrastructure. See the Hudson wiki page for more information about the Hudson installation at Apache. The wiki page also contains instructions on how to get an account if you're a member of the Jackrabbit Team and want to modify the continuous integration settings.

+ +

Build targets and schedule

+ +

See the Jackrabbit page on the Hudson server for the list of configured Jackrabbit builds and their current status.

+ +

The builds are configured to hourly check the Subversion server for updates and to run builds whenever there are changes. Build and test errors are reported on the dev@ mailing list.

+
+
+
+ +
+ + + + + Added: websites/staging/jackrabbit/trunk/content/JCR/creating-releases.html ============================================================================== --- websites/staging/jackrabbit/trunk/content/JCR/creating-releases.html (added) +++ websites/staging/jackrabbit/trunk/content/JCR/creating-releases.html Mon Dec 31 12:40:56 2012 @@ -0,0 +1,259 @@ + + + + + + + Creating Releases + + + +
+ +
+ +
+

Creating Releases

+

This is a how to document for creating Apache Jackrabbit releases. It documents the current release process and needs to be updated as we move forward.

+ +

Release planning

+ +

Jackrabbit releases are created based on user demand and the availability of fixes and other requested changes. Any committer can declare their plan to cut a release by sending a "Apache Jackrabbit x.y.z release plan" message to the dev@ list. The plan should refer to Jira for the list of fixes to be included in the release and give a rough estimate of the release schedule. It's OK to revise the plan if needed.

+ +

If you're not a committer, you can send a message to the mailing list asking for a new release to be made. Including the list of specific fixes you need and a short rationale of why you need the release.

+ +

Prerequisites for release managers

+ +

You need to be a Jackrabbit committer to prepare and perform a release, but anyone is welcome to help test the release candidates and comment on the release plans.

+ +

You should have a code signing key that is included in the Jackrabbit KEYS file. See Appendix A at the end of this page for more details.

+ +

You also need to tell Maven your Subversion credentials needed for deploying artifacts to the Nexus server at https://repository.apache.org/. See
+Appendix B for the required settings.

+ +

Release management tasks

+ +
    +
  1. Make sure that an appropriate version for the release is entered in Jira and that all the related issues have been resolved.
  2. +
  3. Create a RELEASE-NOTES.txt file in the root folder of the project to be released. If such a file already exists, update it for the release. When done, commit the file. See previous release notes for examples of what to include. The release note report in Jira is a useful source of required information.
  4. +
  5. Build and deploy the release artifacts with Maven. See below for the exact steps.
  6. +
  7. Close the staged repository on repository.apache.org.
  8. +
  9. Upload the artifacts to your account on people.apache.org (instructions at the end of the build) +
    +
    +  scp -r target/checkout/target/2.2.12 <username>@people.apache.org:public_html/jackrabbit/
    +  
    +
  10. +
  11. Start the vote thread, wait 72 hours. See the vote template generated by the Maven build.
  12. +
  13. If the vote fails (easy case first) remove the tag from svn and drop the staged repository - done
  14. +
  15. If the vote is successful +
      +
    1. close the vote by publishing the results
    2. +
    3. copy the release archives to the dist directory on www.apache.org (/www/www.apache.org/dist/jackrabbit/), delete the old release there (it's automatically archived), +
      +
      +   # assuming that the release version is 2.2.12, these are the following steps you need to go through
      +   # move release archives to dist
      +   mv ~/public_html/jackrabbit/2.2.12 /www/www.apache.org/dist/jackrabbit
      +   cd /www/www.apache.org/dist/jackrabbit
      +   # change group ownership to jackrabbit user, and allow it 'write' right
      +   chgrp -R jackrabbit /www/www.apache.org/dist/jackrabbit/2.2.12
      +   chmod g+w 2.2.12
      +   # delete old release
      +   rm -rf 2.2.11
      +   
      +
    4. +
    5. release the staged repository for synchronization to Maven central.
    6. +
    7. mark the version as released in Jira: Jira Project Home -> Project Summary -> Administer Project. Under Versions, click <More>. You'll see all the defined project versions. From the settings menu, choose 'Release' on the version.
    8. +
    9. close all the issues included in the release: Jira Project Home -> Change Log -> Choose the released version. From the issue list you have the option to bulk update all of the included issues. Just 'Transition Issues' from 'Resolved' to 'Closed' and you are done!
    10. +
    +
  16. +
  17. Update the Jackrabbit web site to point to the new release.
  18. +
  19. Send the release announcement once the web site and download mirrors have been synced.
  20. +
+ + +

+

Steps to build the release artifacts

+ +

The release is built using the Maven release plugin. See the Releasing a Maven project guide for more details. Make sure you have added the pgp key information in you maven settings file, especially if you have more than one key installed locally. See Appendix B for the details.

+ +
    +
  1. Execute mvn release:prepare. This will update the POM files and tag the release in svn.
  2. +
  3. Execute mvn release:perform. This will build the tagged release and deploy the artifacts to a staging repository on repository.apache.org. The non-Maven release artifacts are automatically deployed to your home directory on people.apache.org. You only need to add the keyname if you have multiple keys and the code signing keys is not your default key.
  4. +
+ + +

Related Links

+ + + + +

+

Appendix A: Create and add your key to the Jackrabbit KEYS file

+ +

Follow these instructions to generate your code signing key and to add it to the Jackrabbit KEYS file.

+ +
    +
  1. Generate a code signing key using your @apache.org address as the email and "CODE SIGNING KEY" as the comment.
  2. +
  3. The Jackrabbit KEYS file is managed in https://svn.apache.org/repos/asf/jackrabbit/dist/KEYS. To modify the file, first checkout the dist directory: +
    +
    +svn checkout https://svn.apache.org/repos/asf/jackrabbit/dist
    +
    +
  4. +
  5. See the beginning of the KEYS file for instructions on how to append your key to the file.
  6. +
  7. Once you've committed the changes, update the KEYS file on people.apache.org: +
    +
    +umask 002; svn update /www/www.apache.org/dist/jackrabbit
    +
    +
  8. +
  9. You are DONE, but to see the changes on http://www.apache.org/dist/jackrabbit/KEYS you must wait 2 hours
  10. +
+ + +

You should get your key linked to the Apache web of trust. Once other people have signed your key, you can update the KEYS file with the signatures you've received.

+ +

+

Appendix B: Maven settings

+ +
+
+<settings>
+...
+  <servers>
+    <!-- To deploy a Jackrabbit snapshot -->
+    <server>
+      <id>apache.snapshots.https</id>
+      <username> <!-- YOUR APACHE SVN USERNAME --> </username>
+      <password> <!-- YOUR APACHE SVN PASSWORD --> </password>
+    </server>
+    <!-- To stage a Jackrabbit release -->
+    <server>
+      <id>apache.releases.https</id>
+      <username> <!-- YOUR APACHE SVN USERNAME --> </username>
+      <password> <!-- YOUR APACHE SVN PASSWORD --> </password>
+    </server>
+    ...
+    <profiles>
+      <profile>
+        <id>apache-release</id>
+        <properties>
+          <gpg.keyname><!-- enough of the key id to id it --></gpg.keyname>
+
+          <!-- pick one of the following -->
+
+          <!-- either you feel comfortable with the passphrase on disk -->
+          <gpg.passphrase><!-- your passphrase for your gpg key goes here--></gpg.passphrase>
+
+          <!-- or you use an agent-->
+          <gpg.useagent>true</gpg.useagent>
+
+        </properties>
+      </profile>
+      ...
+    </profiles>
+  </servers>
+</settings>
+
+
+
+
+
+ +
+ + + + + Added: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-1.png ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-1.png ------------------------------------------------------------------------------ svn:mime-type = image/png Added: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-1.png.jpeg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-1.png.jpeg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-2.png ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-2.png ------------------------------------------------------------------------------ svn:mime-type = image/png Added: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-2.png.jpeg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-2.png.jpeg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-3.png ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-3.png ------------------------------------------------------------------------------ svn:mime-type = image/png Added: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-3.png.jpeg ============================================================================== Binary file - no diff available. Propchange: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.data/deploy-3.png.jpeg ------------------------------------------------------------------------------ svn:mime-type = image/jpeg Added: websites/staging/jackrabbit/trunk/content/JCR/deployment-models.html ============================================================================== --- websites/staging/jackrabbit/trunk/content/JCR/deployment-models.html (added) +++ websites/staging/jackrabbit/trunk/content/JCR/deployment-models.html Mon Dec 31 12:40:56 2012 @@ -0,0 +1,173 @@ + + + + + + + Deployment Models + + + +
+ +
+ +
+

Deployment Models

+

JSR-170 explicitly allows for numerous different deployment models, meaning that it is entirely up to the repository implementation to suggest certain models.

+ +

Jackrabbit is built to support a variety of different deployment models, some of the possibilities on how to deploy Jackrabbit will be outlined here.

+ +

See also the following HOWTO documents for setting up and using the different deployment models:

+ + + + +

Model 1: The (Web-) Application Bundle

+ +

For many applications, usually applications that run in a closed context without interacting with other applications or data sources, it might be desirable to bundle a content repository with the application itself.

+ +

Jackrabbit is built for this lightweight model and allows obviously through the abstraction provided by JSR-170 to move at any point in time to a different deployment model in case this should be desirable for the context that the application runs in.

+ +

This makes Jackrabbit ideal to be packaged as a lightweight out-of-the-box content repository that allows an application vendor to make sure that there are no dependencies to a pre-installed content repository.

+ +

The instance of Jackrabbit that is package with the application is running in-proc inside the same JVM and cannot be accessed by any other application.

+ +

This deployment model is particularly lightweight and does not require a network layer.

+ +

The individual repository instances are started and stopped with their containing applications, which means that the application is not only connecting to the repository but is also in charge of starting and stopping the repository.

+ +

As an example of this deployment model we assume a WebApplication packaged into a .war file which is deployed into a WebContainer, which not only contains the application but also the actual content repository.

+ +
+ + +

Application1 and Application2 both contain their own instances of a Content Repository distributed as a part of their .war file and therefore loaded with the web application's class loader which makes it invisible to other applications.

+ +

This deployment model of course also works for any stand-alone application and not just for web applications.

+ +

Model 2: Shared J2EE Resource

+ +

A second way to deploy a repository is to make it visible as a resource to all the web applications that are running inside a Servlet Container by registering the Repository as a Resource Adapter to the Application Server.

+ +

Similar to the first deployment model this deployment model does also not require a network layer and therefore would be considered in-proc and is running inside the same JVM.

+ +

The repository is started and stopped with the Application Server but is visible to all the applications to connect to.

+ +
+ + +

This setup also allows to take advantage of the XA facilities of the Application Server and could use the application servers single sign-on mechanisms that are provided as part of the J2EE framework.

+ +

Model 3: The Repository Server

+ +

In enterprise environments the client/server deployment model is widely used for relational databases. While with relational databases this is probably the only deployment model that is supported by most RDBMS vendors for repositories, in particular for Jackrabbit, this is only on various options.

+ +

The client/server deployment model will certainly be very popular in environments that where it is desirable to physically separate the content repository (or data) layer from the application or presentation layer, so the content repository can be used from many different applications, physically and can be scaled individually.

+ +
+ + +

This deployment model assumes that there is a network layer between the content repository server and the respective content repository client.

+ +

While the applications communicate through the JSR-170 API with the content repository client. The content repository client then communicates through any form of transport protocol with the stand-alone content repository server.

+ +

The transport protocol between the client and the server is subject to implementation and not mandated by JSR-170. Popular choices might include RMI over JRMP or IIOP or WebDAV and its extensions.

+
+
+
+ +
+ + + + + Added: websites/staging/jackrabbit/trunk/content/JCR/documentation.html ============================================================================== --- websites/staging/jackrabbit/trunk/content/JCR/documentation.html (added) +++ websites/staging/jackrabbit/trunk/content/JCR/documentation.html Mon Dec 31 12:40:56 2012 @@ -0,0 +1,122 @@ + + + + + + + Documentation + + + +
+ + + +
+ + + + +