lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon Willnauer <simon.willna...@googlemail.com>
Subject Re: deleting with sorting and max document
Date Wed, 14 Sep 2011 17:32:54 GMT
can you provide your query which yields all the documents that you
want to delete? I don't understand how the sort order changes anything
here. if you want to only delete the top N docs of that query you
should maybe modify your query to only return those. I could imagine
you are returning the oldest first, if so can't you do a range filter
on top instead of sorting?

simon

On Wed, Sep 14, 2011 at 5:06 PM, Ian Lea <ian.lea@gmail.com> wrote:
> Could never have told that from what you wrote ...
>
> Anyway, I've no idea, beyond 2 general statements:
>
> 1. Lucene is amazingly fast.
> 2. Doing anything on large numbers of docs is liable to take some time.
>
> Make of that what you will.  If it was me, I'd test it.
>
>
> --
> Ian.
>
>
> On Wed, Sep 14, 2011 at 3:28 PM,  <v.sevel@lombardodier.com> wrote:
>> Hi,
>>
>> this was clear actually. I was questionning the performance impact to call
>> IndexReader.deleteDocument(int docNum) one million time. any information
>> about that?
>>
>> thanks,
>> vincent
>>
>>
>>
>>
>>
>>
>>
>> Ian Lea <ian.lea@gmail.com>
>>
>>
>> 14.09.2011 16:20
>> Please respond to
>> java-user@lucene.apache.org
>>
>>
>>
>> To
>> java-user@lucene.apache.org
>> cc
>>
>> Subject
>> Re: deleting with sorting and max document
>>
>>
>>
>>
>>
>>
>> You don't do it with deleteDocuments() but with
>> IndexReader.deleteDocument(int docNum), as I said.
>>
>> To spell it out a bit more:
>>
>> Execute a search to get a list of document ids,
>> searching/sorting/whatever as you wish.
>>
>> Then loop through that list of document ids calling
>> IndexReader.deleteDocument(int docNum) for each one.
>>
>>
>> Hope that's clearer.
>>
>>
>> --
>> Ian.
>>
>>
>> On Wed, Sep 14, 2011 at 3:13 PM,  <v.sevel@lombardodier.com> wrote:
>>> Hi, thanks for your answer. out of the 35 millions docs, I need to
>> delete
>>> 1 million...
>>>
>>> and unfortunately, the ability to put a sort and a max event is not on
>> the
>>> query, but as args in the index searcher.
>>>
>>> so I do not see how to do it with deleteDocuments.
>>>
>>> regards,
>>>
>>> vincent
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Ian Lea <ian.lea@gmail.com>
>>>
>>>
>>> 14.09.2011 15:37
>>> Please respond to
>>> java-user@lucene.apache.org
>>>
>>>
>>>
>>> To
>>> java-user@lucene.apache.org
>>> cc
>>>
>>> Subject
>>> Re: deleting with sorting and max document
>>>
>>>
>>>
>>>
>>>
>>>
>>> You can get a list of document ids via your search/sort call and call
>>> IndexReader.deleteDocument(int docNum) for each one.
>>>
>>>
>>> --
>>> Ian.
>>>
>>>
>>> On Wed, Sep 14, 2011 at 2:23 PM,  <v.sevel@lombardodier.com> wrote:
>>>> Hi,
>>>>
>>>> I have an index with 35 millions docs in it. every day I need to delete
>>>> some of the oldest docs that meet some criteria.
>>>>
>>>> I can easily do this on the searcher by using search(Query query, int
>> n,
>>>> Sort sort)
>>>>
>>>> but there is nothing equivalent for the deleteDocuments.
>>>>
>>>> what are my options?
>>>>
>>>> thanks,
>>>>
>>>> vincent
>>>>
>>>> ************************ DISCLAIMER ************************
>>>> This message is intended only for use by the person to
>>>> whom it is addressed. It may contain information that is
>>>> privileged and confidential. Its content does not
>>>> constitute a formal commitment by Lombard Odier
>>>> Darier Hentsch & Cie or any of its branches or affiliates.
>>>> If you are not the intended recipient of this message,
>>>> kindly notify the sender immediately and destroy this
>>>> message. Thank You.
>>>> *****************************************************************
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>>
>>>
>>> ************************ DISCLAIMER ************************
>>> This message is intended only for use by the person to
>>> whom it is addressed. It may contain information that is
>>> privileged and confidential. Its content does not
>>> constitute a formal commitment by Lombard Odier
>>> Darier Hentsch & Cie or any of its branches or affiliates.
>>> If you are not the intended recipient of this message,
>>> kindly notify the sender immediately and destroy this
>>> message. Thank You.
>>> *****************************************************************
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>>
>>
>> ************************ DISCLAIMER ************************
>> This message is intended only for use by the person to
>> whom it is addressed. It may contain information that is
>> privileged and confidential. Its content does not
>> constitute a formal commitment by Lombard Odier
>> Darier Hentsch & Cie or any of its branches or affiliates.
>> If you are not the intended recipient of this message,
>> kindly notify the sender immediately and destroy this
>> message. Thank You.
>> *****************************************************************
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message