lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amin Mohammed-Coleman <ami...@gmail.com>
Subject Re: Batch Operation and Commit
Date Thu, 26 Aug 2010 19:08:01 GMT
Hi Erick

Thanks for your response.  I used the Lucene in Action 1st edition as a reference for batch
indexing. I've just got my copy of the 2nd edition which mentions that there is no point in
using RAM directory.  Not saying I don't trust you :).

I'll update my code to use the normal fs directory for batch.


Thanks
Amin

On 26 Aug 2010, at 19:33, Erick Erickson wrote:

> I'm going to sidestep your question and ask why you're using
> a RAMDirectory in the first place. People often think it'll
> speed up their indexing because it's in RAM, but the
> normal FS-based indexing caches in RAM too, and you
> can use various settings governing segments, ramusage
> etc. to control how often you flush to disk. So unless you're
> certain you need to, I'd just forget the whole RAM thing <G>.
> 
> You must close your indexwriter OR commit the changes
> before you can see your changes, see IndexWriter.close/commit
> 
> Best
> Erick
> 
> 
> 
> On Thu, Aug 26, 2010 at 10:42 AM, Amin Mohammed-Coleman <aminmc@gmail.com>wrote:
> 
>> Hi
>> 
>> 
>> I have a list of batch tasks that need to be executed.  Each batch contains
>> 1000 documents and basically I use a RAMDirectory based index writer, and at
>> the end of adding 1000 documents to the memory  i perform the following:
>> 
>>   ramWriter.commit();
>>   indexWriter.addIndexesNoOptimize(ramWriter.getDirectory());
>>   ramWriter.close();
>> 
>> 
>> 
>> Do I then need to explicitly do an indexWriter.commit()?  It seems as
>> though if I don't do an explicit commit the documents aren't added to the
>> index (I've inspected via Luke).  I would've thought that the
>> indexWriter.addIndexesNoOptimize would not require me to call the commit
>> explicitly.  Is this a correct assumption? or should i call the commit
>> explicitly for my disk based index writer?
>> 
>> The main idea behind this is that each batch can be executed in a seperate
>> thread and there is only on shared index writer.
>> 
>> Any help would be appreciated.
>> 
>> 
>> Thanks
>> Amin
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 
>> 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message