lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <luc...@mikemccandless.com>
Subject Re: Efficient delete
Date Sat, 16 Aug 2008 08:33:15 GMT
It saves you the hassle of closing your IndexWriter, opening an
IndexReader, doing deletes, closing the IndexReader, then opening a
new IndexWriter.

Also, it can be more efficient (depends on your application) since it
may buffer the deletes for longer than you would if you used
IndexReader.

Mike

Michael Zehrer <zehrer@zepan.org> wrote:
> Mike,
>
> what is the difference if I would use IndexWriter?
>
> Michael
>
> On Fri, Aug 8, 2008 at 12:27, Michael McCandless
> <lucene@mikemccandless.com> wrote:
>>
>> Also, can you do your deletes via IndexWriter (delete by Term) instead of
>> opening IndexReader to do the deletes?
>>
>> Mike
>>
>> Ian Lea wrote:
>>
>>> Michael
>>>
>>>
>>> Did you get anywhere with this? 3 secs for one delete is excessive. A
>>> job of mine ran earlier today and did 2000+ deletes by term on unique
>>> id in less than 9 seconds.  The index is smaller, at around 5Gb, but I
>>> don't believe that would explain the difference.  All the deletes were
>>> done in one batch using lucene 2.3.2.
>>>
>>> Is there a lot of other concurrent activity on the index?  On the
>>> server?  Is it CPU or IO bound?
>>>
>>>
>>> --
>>> Ian.
>>>
>>>
>>>
>>> On Thu, Aug 7, 2008 at 1:46 PM, Michael Zehrer <zehrer@zepan.org> wrote:
>>>>
>>>> Dear List,
>>>>
>>>> I have a rather big index around 20gb. My documents have a unique id
>>>> that I store in in an untokenized field.
>>>>
>>>> Using an IndexReader I delete documents by term using the id. The
>>>> applications tries to batch as many delete operations as possible for
>>>> this.
>>>>
>>>> The applications runs on a 8Core Machine with 1GB heapsize for the
>>>> JVM. Each delete takes about 3sec to complete. I have a lot of Updates
>>>> during a day so this takes a lot of time and in effect this is too
>>>> slow.
>>>>
>>>> Is there anything I can do to speed things up a little?
>>>>
>>>> Best,
>>>>
>>>> Michael
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message