Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (asf.osuosl.org: local policy)
Message-ID: <44BEA8CB.2020604@mikemccandless.com>
Date: Wed, 19 Jul 2006 17:48:59 -0400
From: Michael McCandless <lucene@mikemccandless.com>
User-Agent: Thunderbird 1.5.0.4 (X11/20060516)
MIME-Version: 1.0
To: java-user@lucene.apache.org
Subject: Re: NFS/iSCSI SAN with Lucene
References: <13F36D2926AD444697093282F20041B704A8BDC4@buf-ex01.cymfony.com>
In-Reply-To: <13F36D2926AD444697093282F20041B704A8BDC4@buf-ex01.cymfony.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit


> I did a search on the Lucene list archives, found a lot of posts about
> the use of Lucene with NFS and how there are locking issues, but don't
> see anybody coming to a real solution to this.

We are trying to fix this.  Many people seem to hit it.

The current plan is to first decouple the Locking implementation from 
the Directory implementation:

   http://issues.apache.org/jira/browse/LUCENE-305

and then create a Locking implementation that uses native OS locks 
instead of the simple file-existence locks (used now) that do not work 
with NFS.

But the development is still in process and so it won't be until a 
future Lucene release that this is fixed correctly.

A very good workaround is to use or be inspired by the approach that the 
Solr project:

   http://incubator.apache.org/solr/features.html
   http://incubator.apache.org/solr/tutorial.html

uses, described here:

   http://wiki.apache.org/solr/CollectionDistribution

The gist is, your single Java process that has an IndexWriter 
periodically snapshots the index at a known-safe time, and then the 
multiple Searchers switch to the latest snapshot once the snapshot is 
complete.

Solr actually distributes copies of the index to each Searcher's local 
storage, but you could probably modify this approach so that, instead, 
the Searchers all share a single copy off your SAN.

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org