lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <>
Subject Re: How can we know if 2 lucene indexes are same?
Date Sat, 06 Sep 2008 10:14:20 GMT

I think this may also be useful for a reader to have richer logic as  
to whether it's time to reopen.  It's a way for the writer to do some  
minimal communication to the readers on what changes were just  

Ie we have the static IndexReader.getVersion method, that opens the  
latest segments_N in the index and returns its version; we'd add  
something like static IndexReader.getLatestCommitUserData which  
IndexReader could consult to see if it should reopen.

In 3.0, when IndexWriter hardwires autoCommit to false, then every  
commit is explicitly done by application code (vs now where merges,  
optimize, may do commits as well), then the application would have  
total control on what userData to put into the commit.

I think this is compelling?  It's a tiny change to the index format.


markharw00d wrote:

>> I think this could be a generally useful feature?
> +1. I could definitely use a "commitUserData" option for the same  
> reasons.
>> Thinking more on this, we may not need to modify the index format  
>> at all for
>> this use-case. This is easily achieved in the current system by  
>> adding a
>> dummy document which Solr can read/write -- not very elegant but it  
>> can work
> I thought about this but was uncomfortable with the idea of adding  
> an extra doc - some use cases that become troublesome for any  
> application logic are:
> 1) IndexReader.numDocs/IndexReader.maxDoc will give "incorrect" values
> 2) Any queries of the type "all documents *without* a value for  
> field X return the commit.userdata document.
> I was toying with the idea of maintaining my own commit.userdata  
> file which I would manage in my framework when calling  
> IndexWriter.commit but this does not feel as clean as Lucene core  
> code holding the user data in the segments file.
> Cheers
> Mark
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message