jackrabbit-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Jackrabbit Wiki] Update of "Composite Blob Store Storage Filters" by MattRyan
Date Tue, 15 Aug 2017 21:14:13 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Jackrabbit Wiki" for change notification.

The "Composite Blob Store Storage Filters" page has been changed by MattRyan:
https://wiki.apache.org/jackrabbit/Composite%20Blob%20Store%20Storage%20Filters

New page:
= Composite Blob Store Storage Filters =

NOTE:  The current status of this component is a '''rejected feature''' for Oak 1.8, and a
'''possible future feature''' for later Oak versions.

== Overview ==
Storage filters add capability to [[Composite Blob Store]] by allowing configuration to define
criteria that restrict which blobs can be stored in a composite blob store delegate.  Storage
filters would then be an optional element of delegate configuration.  Delegates would not
be required to have any storage filters at all.

This is a possible future feature that can add a lot of value to the composite blob store
concept.  A major hurdle to this implementation is that JCR node information is not provided
to data store implementations; the normal case is that a binary is created before the corresponding
node.  This problem has to be resolved in order to implement this feature.  More discussion
on this below.

== Technical Details ==
=== Storage Filter Types ===
Storage filters operate on any combination of the following:
 * JCR path
 * JCR node type
 * Existence of JCR property
 * JCR property value

=== Impacts on Composite Blob Store ===
The existence of a storage filter on a delegate blob store may have an impact on the delegate
traversal strategy implementation.  For example, an implementation may consider that a blob
store with filters should take precedence over a blob store without filters in terms of choosing
a write destination.

==== Impact on Intelligent Delegate Traversal Strategy ====
If storage filters were added to composite blob store, it is expected that the following changes
would be added to the Intelligent Delegate Traversal Strategy, which is the default traversal
strategy.

===== Delegate Search Order =====
Speaking generally, delegates with storage filters would take highest precedence, and would
be tried first for reads and writes.

===== Writes =====
Delegates that have storage filters always have write precedence over delegates that do not
have storage filters - meaning writes always happen to delegates with filters if the filters
match what is being written.

The algorithm for writing a blob to the composite blob store would be changed:
 * When determining where to write the blob, start with delegates that have storage filters.
 Prefer writing to a delegate with a storage filter that matches the blob being written.
 * When searching for other versions of the same blob, '''also''' search using delegates that
have filters that don't match the blob being written (after searching delegates without filters).
   * Configuration change can cause this situation - a blob may originally have been written
to a blob store, and afterward configuration changes such that the blob would not have been
written to that blob store originally. See "Using Non-matching Delegates" below for more.
   * If this case occurs, the code should also asynchronously remove the blob from the incorrect
location once it has been written to the correct location.

===== Reads =====
Delegates that have storage filters always have read precedence over delegates that do not
have storage filters - meaning we always attempt to read from them first, because matching
filters is faster than checking a delegate to see if a blob exists.

The algorithm for reading a blob from the composite blob store would be changed:
 * When searching for a blob, start with delegates that have storage filters that match the
blob.
 * If no match, search delegates without storage filters.
 * If no match, search delegates that have storage filters that do not match the blob.
   * Configuration change can cause this situation - a blob may originally have been written
to a blob store, and afterward configuration changes such that the blob would not have been
written to that blob store originally. See "Using Non-matching Delegates" below for more.
   * If this case occurs, the code should also asynchronously move the blob from the incorrect
location to the correct location.

===== Using Non-matching Delegates =====

This step is necessary to handle situations where blobs are temporarily located in the "wrong"
blob store - in other words, when a blob is located in a delegate where it would not be written
according to configuration. The most obvious case where this could occur is in the case of
configuration change. A delegate D may be configured with certain storage filters, causing
Blob B to be written there. Then the configuration is changed such that if B were being written
now it would not have been written to D. This final step allows B to be found in D even though
it doesn't match the storage filters.

When this situation is encountered, the composite blob store should also initiate an asynchronous
background job to move the blob from it's current location to the proper one - the location
where it would be found if it were being created now - on read requests, or for write requests
should write to the correct location and remove the blob asynchronously in the background
from the current location after the write is done.

Mime
View raw message