lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Emir Arnautović <emir.arnauto...@sematext.com>
Subject Re: Very high number of deleted docs
Date Wed, 04 Oct 2017 13:03:06 GMT
Hi Markus,
It is passed but not explicitly - it uses reflection to pass arguments - take a look at parent
factory class.

When it comes to force merging - you have extreme case - 80% is deleted (my guess frequent
updates) and extreme cases require some extreme measures - it can be either periodic force
merge or full reindexing + aliases.

HTH,
Emir

--
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/



> On 4 Oct 2017, at 14:47, Markus Jelsma <markus.jelsma@openindex.io> wrote:
> 
> Do you mean a periodic forceMerge? That is usually considered a bad habit on this list
(i agree). It is just that i am actually very surprised this can happen at all with default
settings. This factory, unfortunately does not seem to support settings configured in solrconfig.
> 
> Thanks,
> Markus
> 
> -----Original message-----
>> From:Amrit Sarkar <sarkaramrit2@gmail.com>
>> Sent: Wednesday 4th October 2017 14:42
>> To: solr-user@lucene.apache.org
>> Subject: Re: Very high number of deleted docs
>> 
>> Hi Markus,
>> 
>> Emir already mentioned tuning *reclaimDeletesWeight which *affects segments
>> about to merge priority. Optimising index time by time, preferably
>> scheduling weekly / fortnight / ..., at low traffic period to never be in
>> such odd position of 80% deleted docs in total index.
>> 
>> Amrit Sarkar
>> Search Engineer
>> Lucidworks, Inc.
>> 415-589-9269
>> www.lucidworks.com
>> Twitter http://twitter.com/lucidworks
>> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
>> 
>> On Wed, Oct 4, 2017 at 6:02 PM, Emir Arnautović <
>> emir.arnautovic@sematext.com> wrote:
>> 
>>> Hi Markus,
>>> You can set reclaimDeletesWeight in merge settings to some higher value
>>> than default (I think it is 2) to favor segments with deleted docs when
>>> merging.
>>> 
>>> HTH,
>>> Emir
>>> --
>>> Monitoring - Log Management - Alerting - Anomaly Detection
>>> Solr & Elasticsearch Consulting Support Training - http://sematext.com/
>>> 
>>> 
>>> 
>>>> On 4 Oct 2017, at 13:31, Markus Jelsma <markus.jelsma@openindex.io>
>>> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> Using a 6.6.0, i just spotted one of our collections having a core of
>>> which over 80 % of the total number of documents were deleted documents.
>>>> 
>>>> It has <mergePolicyFactory class="org.apache.solr.index.TieredMergePolicyFactory"/>
>>> configured with no non-default settings.
>>>> 
>>>> Is this supposed to happen? How can i prevent these kind of numbers?
>>>> 
>>>> Thanks,
>>>> Markus
>>> 
>>> 
>> 


Mime
View raw message