lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jong Kim <jong.luc...@gmail.com>
Subject Re: Lucene index on NFS
Date Tue, 02 Oct 2012 13:26:49 GMT
My Lucene index is accessed by multiple threads in a single process.

/Jong

On Tue, Oct 2, 2012 at 8:45 AM, Paul Libbrecht <paul@hoplahup.net> wrote:

> I doubt NFS is an unreliable file-system.
> Lucene uses normal random access to files and this has no reason to be
> unreliable unless bad things such as network drops happen (in which case
> you'd get direct failures or  timeouts rather than corruption). I've seen
> fairly large infrastructures being based on NFS and corruption is something
> I've never heard about.
>
> Note: no concurrent access to a lucene index, right?
>
> Paul
>
>
> Le 2 oct. 2012 à 14:01, Jong Kim a écrit :
>
> > Thank you all for reply.
> >
> > So it soudns like it is a known fact that the performance would suffer
> > rather significantly when the index files are accessed over NFS. But how
> > about reliability and robustness (which seems even more important)? Isn't
> > there any increased possibility for intermittent errors such as index
> file
> > corruption (due to cache inconsistency, difference in delete semantics,
> > etc.) when using NFS? Has anyone run into such trouble? Or is it strictly
> > just a performance issue?
> >
> > /Jong
> > On Tue, Oct 2, 2012 at 5:17 AM, Paul Libbrecht <paul@hoplahup.net>
> wrote:
> >
> >> My experience in the Lucene 1.x times were a factor of at least four in
> >> writing to NFS and about two when reading from there. I'd discourage
> this
> >> as much as possible!
> >>
> >> (rsync is way more your friend for transporting and replication à la
> solr
> >> should also be considered)
> >>
> >> paul
> >>
> >>
> >> Le 2 oct. 2012 à 11:10, Ian Lea a écrit :
> >>
> >>> You'll certainly need to factor in the performance of NFS versus local
> >> disks.
> >>>
> >>> My experience is that smallish low activity indexes work just fine on
> >>> NFS, but large high activity indexes are not so good, particularly if
> >>> you have a lot of modifications to the index.
> >>>
> >>> You may want to install a custom IndexDeletionPolicy.  See the
> >>> javadocs for details with specific reference to NFS.
> >>>
> >>>
> >>> --
> >>> Ian.
> >>>
> >>> On Tue, Oct 2, 2012 at 3:21 AM, Vitaly Funstein <vfunstein@gmail.com>
> >> wrote:
> >>>> How tolerant is your project of decreased search and indexing
> >> performance?
> >>>> You could probably write a simple test that compares search and write
> >>>> speeds of local and NFS-mounted indexes and make the decision based
on
> >> the
> >>>> results.
> >>>>
> >>>> On Mon, Oct 1, 2012 at 3:06 PM, Jong Kim <jong.lucene@gmail.com>
> wrote:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> According to the Lucene In Action (Second Edition), the section
> 2.11.2
> >>>>> "Accessing an index over a remote file system" explains that there
> are
> >>>>> issues related to accessing a Lucene index across remote file system
> >>>>> including NFS.
> >>>>>
> >>>>> I'm particuarly interested in NFS compatibility, and wondering if
> >> there has
> >>>>> been any work done to solve or mitigate this problem. Has this issue
> >> been
> >>>>> addressed? If not, are there some reliable work-arounds that make
> this
> >>>>> possible at the expense of some sacrifice in other areas?
> >>>>>
> >>>>> Any information would be greatly appreciated, since my project
> heavily
> >>>>> depends on the feasibility of this.
> >>>>>
> >>>>> Thanks
> >>>>> /Jong
> >>>>>
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message