Mailing-List: contact java-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: java-user@lucene.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-ID: <1f8734680808160009p3cf159ob012037f35c1be89@mail.gmail.com>
Date: Sat, 16 Aug 2008 09:09:59 +0200
From: "Michael Zehrer" <zehrer@zepan.org>
Sender: michael@zepan.net
To: java-user@lucene.apache.org
Subject: Re: Efficient delete
In-Reply-To: <36F19444-085E-4001-9C6A-D8C06302B062@mikemccandless.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <1f8734680808070546q6d36cbe8m6687b70f7782d6df@mail.gmail.com>
	 <8c4e68610808080223p343f390i1fc3d2a5e42b6a7e@mail.gmail.com>
	 <36F19444-085E-4001-9C6A-D8C06302B062@mikemccandless.com>

Mike,

what is the difference if I would use IndexWriter?

Michael

On Fri, Aug 8, 2008 at 12:27, Michael McCandless
<lucene@mikemccandless.com> wrote:
>
> Also, can you do your deletes via IndexWriter (delete by Term) instead of
> opening IndexReader to do the deletes?
>
> Mike
>
> Ian Lea wrote:
>
>> Michael
>>
>>
>> Did you get anywhere with this? 3 secs for one delete is excessive. A
>> job of mine ran earlier today and did 2000+ deletes by term on unique
>> id in less than 9 seconds.  The index is smaller, at around 5Gb, but I
>> don't believe that would explain the difference.  All the deletes were
>> done in one batch using lucene 2.3.2.
>>
>> Is there a lot of other concurrent activity on the index?  On the
>> server?  Is it CPU or IO bound?
>>
>>
>> --
>> Ian.
>>
>>
>>
>> On Thu, Aug 7, 2008 at 1:46 PM, Michael Zehrer <zehrer@zepan.org> wrote:
>>>
>>> Dear List,
>>>
>>> I have a rather big index around 20gb. My documents have a unique id
>>> that I store in in an untokenized field.
>>>
>>> Using an IndexReader I delete documents by term using the id. The
>>> applications tries to batch as many delete operations as possible for
>>> this.
>>>
>>> The applications runs on a 8Core Machine with 1GB heapsize for the
>>> JVM. Each delete takes about 3sec to complete. I have a lot of Updates
>>> during a day so this takes a lot of time and in effect this is too
>>> slow.
>>>
>>> Is there anything I can do to speed things up a little?
>>>
>>> Best,
>>>
>>> Michael
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org