lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <j...@apache.org>
Subject [jira] Commented: (LUCENE-305) [PATCH] Lock Framework - allows custom lock mechanism
Date Thu, 29 Jun 2006 16:34:30 GMT
    [ http://issues.apache.org/jira/browse/LUCENE-305?page=comments#action_12418493 ] 

Michael McCandless commented on LUCENE-305:
-------------------------------------------


I'm working towards a patch for this.  I changed the name of the issue
to better reflect the goal.

I started with the above approach and made some changes / brought it
up to current Lucene HEAD sources.

I'd like to sanity check the approach to get any feedback:

  * Create a new abstract base class called LockFactory (like the
    above patch).  Purpose of this class is to provide a makeLock()
    method that returns a Lock object for given name.
    Directory.makeLock just calls this method.

  * Create 3 concrete subclasses (for now):

    * SimpleFSLockFactory (factored out of FSDirectory)

    * SingleProcessLockFactory (factored out of RAMDirectory)

    * NoLockFactory

    * (then, some time soon: NativeFSLockFactory)

  * In Directory, add a setLockFactory() method.  This method is
    optional.  If it is not called, then each Directory has a default
    LockFactory that's used instead.  FSDirectory defaults to either
    SimpleFSLockFactory or NoLockFactory (if setDisabledLocks are
    called) and RAMDirectory defaults to SingleProcessLockFactory.

I believe this approach will be fully backwards compatible with
current locking/APIs:

  * FSDirectory and RAMDirectory (and anyone who subclasses) use the
    same locking they use today.  Same lock file names are used and
    same defaulting of LOCK_DIR is used.

  * If any subclass of Directory has its own makeLock() method, it
    will still fully override all of this new logic.

  * No existing APIs are changed; only new methods are added and new
    classes (*LockFactory).

  * Performance should be the same (I'm trying hard not to cause
    additional code to be executed in the makeLock() code path).  I'll
    test this to make sure I'm right :)

This approach means that a user can instantiate a Directory, any
Directory, and then set its locking implementation like this:

  Directory.setLockFactory(new SingleProcessLockFactory();

So this should make it easy to use an FSDirectory with a SingleProcess
lock, for example.  Users can also subclass their own LockFactory
implementations (eg the MySQL based locking above) and easily use
those for locking.

Please note that the goal here is to make Locking independent of the
Directory, meaning I'm not yet adding any new locking implementations.
The above locking implementations were already pre-existing in Lucene,
just refactored to subclasses of LockFactory.  After this (I'll open a
separate issue), I'd like to build a NativeFSLockFactory to address
the original issues/problems we've seen with the
java.io.File.createNewFile() implementation we have now.

This approach differs from the original approach above in that instead
of a global property that sets lock factory class, you explicity call
setLockFactory for a given Directory instance.  In addition, this
approach factors out all locking logic (except
getDefaultLockFactory()) inside FSDirectory and RAMDirectory into
LockFactory.

Please let me know if anyone has any concerns / issues with this plan!
I have an initial working version that passes all current unit tests,
but I still need to add some more specific test cases for locking, do
performance testing, etc.  Thanks.


> [PATCH] Lock Framework - allows custom lock mechanism
> -----------------------------------------------------
>
>          Key: LUCENE-305
>          URL: http://issues.apache.org/jira/browse/LUCENE-305
>      Project: Lucene - Java
>         Type: Improvement

>   Components: Store
>     Versions: unspecified
>  Environment: Operating System: other
> Platform: All
>     Reporter: Jeff Patterson
>     Assignee: Lucene Developers
>     Priority: Minor
>  Attachments: FSDirectory.java, FSDirectory_patch_file.txt, Lock.java, LockFactory.java,
Lock_patch_file.txt, MySQLLocker.java
>
> Proposal:  Pluggable Lock Framework for Lucene
> Date:      Nov 2004
> Developer: Jeff Patterson (jeffATwebdoyen.com - http://www.webdoyen.com)
> ------
> Abstract:  A framework to allow Lucene users to override the default
> FileSystem locking mechanism with a custom lock mechanism.
> A Lucene user may develop a new class that extends 
> org.apache.lucene.store.Lock and implement bodies for the following
> methods:
>     public boolean obtain()    - to obtain custom lock
>     public boolean isLocked()  - to detect custom lock
>     public void release()      - to release custom lock
> NOTE: When implementing these methods, the developer should make sure to 
> use the this.getLockName() method on the Lock to identify which lock
> is being manipulated (see Modified Files below for more).
> After developed, the new class must be added to the classpath (along
> with any other supporting classes/libraries needed by the new class),
> and the Lucene framework must be alerted of the new class by way of
> the "org.apache.lucene.lockClass" -D System property.  Example:
>    java -Dorg.apache.lucene.lockClass=foo.MyCustomLocker LuceneTest
> ------
> Modified Files:  The following files were modified to support 
> this framework (DIFF files at end):
> - org.apache.lucene.store.Lock
>   The member "lockName" and an accompanying protected getter and
>   setter were added to this class to support naming the lock.  This
>   is transparent to the default lock mechanism and is only useful
>   when writing a custom lock.
> - org.apache.lucene.store.FSDirectory
>   Instead of instantiating a default Lock, this class now checks
>   to see if an overridden Lock mechanism is provided, and if so
>   asks the LockFactory (see below) to provide an overridden Lock
>   class.
> New Files:  The following files were added to support this framework:
> - org.apache.lucene.store.LockFactory
>   This class is used to reflect and instantiate by name the custom
>   Lock implementation.  Error handing should be modified in this
>   class, but that would have required a more extensive code overhaul.
>   The javadocs for the LockFactory contain a skeleton Java file for
>   a custom lock implementation.
> ------

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message