lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Re: Index + Taxonomy Replication
Date Fri, 01 Nov 2013 07:12:52 GMT
SearcherTaxonomyManager can be used only for NRT, as it only takes an
IndexWriter and DirectoryTaxonomyWriter. And I don't think you want to keep
those writers open on the slaves side.

I think that a ReferenceManager, which returns a SearcherAndTaxonomy, is
the right thing to do. The reason why we don't offer it is because it's
very tricky to use outside of a well defined "refresh" protocol. If we let
you "refresh" a Directory-based pair, and you're not careful enough, you
could end up reopening the IndexReader before the TaxonomyReader was
committed, or vice versa. Both lead to unsynchronized IR/TR pair, which is
bad. However, if your app always calls this maybeRefresh once the Handler
is done (i.e. as a Callback), and it is *the only one* that refreshes the
pair, then you're safe.

Maybe we should offer such a ReferenceManager (maybe it can even be
SearcherTaxonomyManager which takes a pair of Directory in another ctor),
and document that its maybeRefresh needs to be called by the same thread
that modified the index (i.e. commit() or replication).

Shai


On Thu, Oct 31, 2013 at 12:53 PM, Michael McCandless <
lucene@mikemccandless.com> wrote:

> Maybe have a look at how the IndexAndTaxonomyReplicationClientTest.java
> works?
>
> Hmm, in its callback, it manually reopens the index + taxoIndex, but I
> think you could instead use a SearcherTaxonomyManager and call its
> .maybeRefresh inside your callback?
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Wed, Oct 30, 2013 at 11:24 AM, Joe Eckard <eckardjf@gmail.com> wrote:
> > Hello,
> >
> > I'm attempting to setup a master/slave arrangment between two servers
> where
> > the master uses a SearcherTaxonomyManger to index and search, and the
> slave
> > is read-only - using just an IndexSearcher and TaxonomyReader.
> >
> > So far I am able to publish new IndexAndTaxonomyRevisions on the master
> and
> > pull them down to the slave with no problems (using the HttpReplicator
> and
> > an IndexAndTaxonomyReplicationHandler), but I'm not sure how to correctly
> > reopen the IndexSearcher and TaxonomyReader pair in the
> > ReplicationHandler's callback.
> >
> > Should I wrap them in some kind of ReferenceManager to allow searches to
> > continue on the read-only server during the cutover? Is there a specific
> > order they should be reopened in?
> >
> > Any advice or pointers would be much appreciated.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message