Mailing-List: contact java-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-dev@lucene.apache.org
Message-ID: <28672514.1161225156789.JavaMail.jira@brutus>
Date: Wed, 18 Oct 2006 19:32:36 -0700 (PDT)
From: "Michael McCandless (JIRA)" <jira@apache.org>
To: java-dev@lucene.apache.org
Subject: [jira] Commented: (LUCENE-678) [PATCH] LockFactory implementation
 based on OS native locks (java.nio.*)
In-Reply-To: <31596290.1159842979597.JavaMail.root@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

    [ http://issues.apache.org/jira/browse/LUCENE-678?page=comments#action_12443394 ] 
            
Michael McCandless commented on LUCENE-678:
-------------------------------------------

OK!  Sorry, you are correct.  Of course on closing the first
IndexWriter cleanly, it will remove its lock files.  Good catch.

Now I remember (and just re-tested) why I added this line: as a test,
I temporarily changed FSDirectory's default locking to
NativeFSLockFactory and then ran all unit tests.  This test case
failed, with IOException on creating that second IndexWriter and my
[wrong] fix was to close the first writer.  But you're right that fix
is not right because it turns off the test entirely.

However, I still think the test is incorrect.

Really what the test is trying to do is "simulate" a crashed writer
and then verify creating a new writer on that directory works.  With
SimpleFSLockFactory, an effective "simulation" is to leave open the
first IndexWriter.  But with NativeFSLockFactory, it's not, because
the OS won't let you remove the lock files (that are still held open)
-- this is actually a good thing because in this context it would let
the programmer know that indeed another writer is still holding the
write lock.

Maybe we should remove the writer.close and then disable this test
when we switch to Native locks as the default [in the future]?  Doron
what do you think?


UGH, I just found a serious [different] problem with my patch.  I'm
doing too much "sharing" for the locks such that different directories
will incorrectly share locks!  Ugh.

I'm reopening this so I can fix it properly.  Sorry about this.


> [PATCH] LockFactory implementation based on OS native locks (java.nio.*)
> ------------------------------------------------------------------------
>
>                 Key: LUCENE-678
>                 URL: http://issues.apache.org/jira/browse/LUCENE-678
>             Project: Lucene - Java
>          Issue Type: New Feature
>          Components: Store
>    Affects Versions: 2.1
>            Reporter: Michael McCandless
>         Assigned To: Yonik Seeley
>            Priority: Minor
>             Fix For: 2.0.1
>
>         Attachments: LUCENE-678-patch.txt
>
>
> The current default locking for FSDirectory is SimpleFSLockFactory.
> It uses java.io.File.createNewFile for its locking, which has this
> spooky warning in Sun's javadocs:
>     Note: this method should not be used for file-locking, as the
>     resulting protocol cannot be made to work reliably. The FileLock
>     facility should be used instead.
> So, this patch provides a LockFactory implementation based on FileLock
> (using java.nio.*).
> All unit tests pass with this patch, on OS X (10.4.8), Linux (Ubuntu
> 6.06), and Windows XP SP2.
> Another benefit of native locks is the OS automatically frees them if
> the JVM exits before Lucene can free its locks.  Many people seem to
> hit this (old lock files still on disk) now.
> I've created this new class:
>   org.apache.lucene.store.NativeFSLockFactory
> and added a couple test cases to the existing TestLockFactory.
> I've left SimpleFSLockFactory as the default locking for FSDirectory
> for now.  I think we should get some usage / experience with
> NativeFSLockFactory and then later on make it the default locking
> implementation?
> I also tested changing FSDirectory's default locking to
> NativeFSLockFactory and all unit tests still pass (on the above
> platforms).
> One important note about locking over NFS: some NFS servers and/or
> clients do not support it, or, it's a configuration option or mode
> that must be explicitly enabled.  When it's misconfigured it's able to
> take a long time (35 seconds in my case) before throwing an exception.
> To handle this, I acquire & release a random test lock on creating the
> NativeFSLockFactory to verify locking is configured properly.
> A few other small changes in the patch:
>     - Added a "failure reason" to Lock.java so that in
>       obtain(lockWaitTimeout), if there is a persistent IOException
>       in trying to obtain the lock, this can be messaged & included in
>       the "Lock obtain timed out" that's raised.
>     - Corrected javadoc in SimpleFSLockFactory: it previously said the
>       wrong system property for overriding lock class via system
>       properties
>     - Fixed unhandled IOException when opening an IndexWriter for
>       create, if the locks dir does not exist (just added
>       lockDir.exists() check in clearAllLocks method of
>       SimpleFSLockFactory & NativeFSLockFactory.
>     - Fixed a few small unrelated issues with TestLockFactory, and
>       also fixed tests to accept NativeFSLockFactory as the default
>       locking implementation for FSDirectory.
>     - Fixed a typo in javadoc in FieldsReader.java
>     - Added some more javadoc for the LockFactory.setLockPrefix

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org