lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthew DeLoria" <matthew.delo...@gmail.com>
Subject Reading from an IndexWriter
Date Mon, 03 Nov 2008 22:24:51 GMT
I had a question about more about Best Practices and reading from an
IndexWriter.

Currently, we have an index which we call the master index. This index, in
itself, represents our data model. Many clients can access this index.

However, we have importer and updating clients which essentially add to this
index periodically. These tasks can have specific logic where we can grab
specific documents, update some of the data, and call
writer.updateDocument(..). We also allow the adding of documents. Each of
these tasks however, may depend on data we are adding to the writer at the
same time.

For example, I could say writer.addDocument() and a second later I may need
to do a query for this very document I just added. Currently, we have a temp
directory where all the writing is occurring. We have a searcher that
searches this index. Now, for this searcher to see the writes that occurring
to this temp index, it needs to be reconstructed each time we need to do a
search which is very very inefficient, as this could happen very frequently.
Consider the situation where I add a document and then need to get this
document immediately after. The searcher would need to be closed and the
reader reopened. I will also have to call a commit (or flush) on the writer
before doing this. Unfortunantly, we can't have our TempDirectory be a ram
directory exclusively because we can't guarantee how much memory each client
will have.

So my question is, is there a way I can read what documents are sitting in
the writer without having to do this painful flush/reopen? I know this is
not how Lucene is intended to work but in our case it would be very very
helpful if we could do the reading and writing from the same
IndexWriter/Reader so we wouldn't have to keep doing this reopen / flush
call.

Second, if nothing like this is possible, is the way I am doing it above the
best possible way - (Calling flush on the writer, calling reopen on the
indexreader, and reconstructing the searcher)

I am using Lucene 2.3.2 currently.

Thanks!
m

-- 
Matthew P. DeLoria
matthew.deloria@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message