lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: How can we know if 2 lucene indexes are same?
Date Fri, 05 Sep 2008 16:22:37 GMT

Shalin Shekhar Mangar wrote:

> On Fri, Sep 5, 2008 at 6:03 PM, Michael McCandless <
> lucene@mikemccandless.com> wrote:
>
>>
>> Large segment merges will also send huge traffic.  You may just  
>> want to
>> send all updates (document adds/deletes) to all slaves directly?   
>> It'd be
>> nice if you could somehow NOT sync the effects of segment merging,  
>> but do
>> sync doc add/deletes... not sure how to do that.
>>
>
> As Noble said, that is another option we can consider.

Well this is certainly a nice challenging problem :)

> Thanks for the explanation Mike. The core problem is to make sure both
> indices are in sync. The log replication helps us because we compare  
> the
> master and slave index with a reference point (log position). If it  
> becomes
> possible for us to specify a version number during a commit, we can  
> use the
> master's version number on the slave. This can help us compare the two
> indices. Not sure if that API change will be generally useful.  
> Thoughts?

I think this could be a generally useful feature?

So you're thinking IndexWriter.commit() would take an optional opaque  
argument (maybe a String for generality?) that's recorded into the  
segments_N and could then later be retrieved by IndexReader and  
IndexWriter?

After a merge completes, should it just carry forward whatever was  
stored on the last segments_N?

We should call it something other than version, which already exists  
-- maybe "commitDetails", "commitComment", "commitUserData" or  
something?

Mike

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message