lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Batch Operation and Commit
Date Thu, 26 Aug 2010 18:33:17 GMT
I'm going to sidestep your question and ask why you're using
a RAMDirectory in the first place. People often think it'll
speed up their indexing because it's in RAM, but the
normal FS-based indexing caches in RAM too, and you
can use various settings governing segments, ramusage
etc. to control how often you flush to disk. So unless you're
certain you need to, I'd just forget the whole RAM thing <G>.

You must close your indexwriter OR commit the changes
before you can see your changes, see IndexWriter.close/commit

Best
Erick



On Thu, Aug 26, 2010 at 10:42 AM, Amin Mohammed-Coleman <aminmc@gmail.com>wrote:

> Hi
>
>
> I have a list of batch tasks that need to be executed.  Each batch contains
> 1000 documents and basically I use a RAMDirectory based index writer, and at
> the end of adding 1000 documents to the memory  i perform the following:
>
>    ramWriter.commit();
>    indexWriter.addIndexesNoOptimize(ramWriter.getDirectory());
>    ramWriter.close();
>
>
>
> Do I then need to explicitly do an indexWriter.commit()?  It seems as
> though if I don't do an explicit commit the documents aren't added to the
> index (I've inspected via Luke).  I would've thought that the
> indexWriter.addIndexesNoOptimize would not require me to call the commit
> explicitly.  Is this a correct assumption? or should i call the commit
> explicitly for my disk based index writer?
>
> The main idea behind this is that each batch can be executed in a seperate
> thread and there is only on shared index writer.
>
> Any help would be appreciated.
>
>
> Thanks
> Amin
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message