lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael McCandless <luc...@mikemccandless.com>
Subject Re: [lucene 4.6] NPE when calling IndexReader#openIfChanged
Date Mon, 19 May 2014 10:19:11 GMT
On Mon, May 19, 2014 at 6:14 AM, Clemens Wyss DEV <clemensdev@mysign.ch> wrote:
> Mike,
> first of all thanks for all your input, I really appreciate (as much as I like reading
your blog).

You're welcome!

>> Hmm, but you swap these files over while an IndexReader is still open on the index?
> no IndexReader is open while swapping. At least not by design. We have at most one (current)reader
per Index, which is closed just before swapping. I will investigate on this more deeply, though,
me not being impeccable ;)

Hmm, double check that.  The only way I can explain what you're seeing
is if the reader was open on the index, then the index was blown away
/ replaced, then openIfChanged called ...

>> Instead, use IndexWriter.deleteAll, then index all your new docs,
>>then commit, then reopen the reader.
> we were facing memory issues and hence decided to do "intermediate commits" (i.e. flush
the writer). And in order to still provide "atomicity" (with respect to a full re-index) we
decided to go the before mentionend approach.

Can you just decrease IW's ramBufferSizeMB to relieve the memory pressure?

>> Separately, it's better to use a near-real-time IndexReader so
>>you don't have to commit in order to see changes.
> What if I were to search right after deleteAll?

That's fine, it's fully transactional: the deleteAll will not be
visible until you open a new reader.  If you change your mind, you can
call rollback after deleteAll and the docs are back, etc.

Even opening a new IW with OpenMode.CREATE is transactional: that
change will not become visible to a reader until it's reopened.

Mike McCandless

http://blog.mikemccandless.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message