lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Desteny Child <myshar...@gmail.com>
Subject Re: Lucene cluster with NFS or synchronization tool such as rsync
Date Mon, 04 Jul 2016 12:10:21 GMT
Hi Mike,

Thanks you very much for your response.

I would be really grateful if you can please provide me with an information
where I can read(may be with examples) about new near-real-time replication
?

Thanks,
Alex

2016-07-04 12:57 GMT+03:00 Michael McCandless <lucene@mikemccandless.com>:

> NFS is dangerous if different nodes may take turns writing to the shared
> index.
>
> Locking sometimes doesn't work correctly, client-side metadata caching
> (e.g. the directory entry) can cause problems, NFS doesn't support "delete
> on final close" semantics that Lucene relies on.
>
> rsync-like behavior can work with IndexWriter if you use
> SnapshotDeletionPolicy to hold a point-in-time view of the index open for
> copying ... this is also how to take a live backup of a still-writing
> index, and it's how Lucene's replication module works.
>
> You could also try the new near-real-time replication, which copies just
> the newly written segment files without requiring a full commit (fsync) on
> the source index.
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
> On Sun, Jul 3, 2016 at 2:09 PM, Desteny Child <myshareit@gmail.com> wrote:
>
> > I need to organize a cluster for my stateless application based on Lucene
> > 5.2.1. Right now I'm looking for a solution in order to share Lucene
> index
> > via NFS or rsync between different Lucene nodes.
> >
> > Is it a good idea to use NFS for this purpose and if so will it be
> possible
> > to read/write from different nodes to the same shared index ?
> >
> > Also I read that rsync tool can be used for this purpose(in order to
> > synchronize index files across all nodes) but I can't find any success
> > story for using rsync + Lucene. Right now I have a lot of question, one
> of
> > them - is it safe to use rsync at anytime especially when IndexWriter is
> in
> > progress(not closed) and actively indexes documents.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message