lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Questions on index Writer
Date Fri, 15 Jul 2011 20:03:02 GMT
Index files should not be disappearing unless you're using the form
of opening an indexwriter that creates a new index. We'd need to see
the code you use top open the IW to provide more help.

If all you're doing is looking at the index directory, segments will disappear
as they are merged so that's also what you might be seeing. Can you give
us more details?

A good time to commit? Well, that depends. It really depends upon your app.
Given what you've described, and assuming that you've set reasonable
numbers for, say, setMaxBufferedDocs, setRAMBufferSizeMB just letting the
commit happen when you close is probably fine.

What do you hope to accomplish by executing maybeMerge()? Personally I'd
stay away from that unless you're solving a specific question and just let the
segment merge process operate to handle all that...

Best
Erick

On Fri, Jul 15, 2011 at 3:03 PM, Saurabh Gokhale
<saurabhgokhale@gmail.com> wrote:
> Hi All,
>
> I have following questions about lucene indexWriter. I am using version
> 3.1.0.
>
> While indexing documents,
> 1. When is the good time to commit changes? (indexWriter.commit) or just
> close the writer after the indexing is done so that commit automatically
> happens.
> 2. When is the good time to merge indexes (indexWriter.maybeMerge()).  Is it
> just before committing the changes or after indexing say X number of
> documents. (I recently upgraded from 2.9.4 to 3.1 and I see 3.1 lucene
> generates lot of small index files while indexing document)
>
> Also I have a problem where my lucene index files sometimes gets deleted
> from the index folder. I am not sure what code snippet is causing the
> existing index files to accidently get removed.
>
> My indexer runs in a thread loop where it indexes file whenever they are
> available. When no more files are available, indexer thread closes the
> writer and goes to sleep, after specific time, it again creates a new index
> on the same folder and starts indexing new files if any available.
>
> A. Is it a wrong way to index files?
> B. Because I close the index and open it again later, am I seeing my lucene
> index files getting deleted?
>
> Thanks
>
> Saurabh
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message