lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Santamauro <david.santama...@gmail.com>
Subject Re: Urgent help on solr optimisation issue !!
Date Fri, 07 Jun 2019 14:09:01 GMT

/clarification/ ... expungeDeletes will merge every segment *touched by the current commit*
that has a deleted document.


On 6/7/19, 10:07 AM, "David Santamauro" <david.santamauro@gmail.com> wrote:

    Erick, on 6.0.1, optimize with maxSegments only merges down to the specified number. E.g.,
given an index with 75 segments, optimize with maxSegments=74 will only merge 2 segments leaving
74 segments. It will choose a segment to merge that has deleted documents, but does not merge
every segment with deleted documents.
    
    I think you are thinking about the expungeDeletes parameter on the commit request. That
will merge every segment that has a deleted document.
    
    
    On 6/7/19, 10:00 AM, "Erick Erickson" <erickerickson@gmail.com> wrote:
    
        This isn’t quite right. Solr will rewrite _all_ segments that have _any_ deleted
documents in them when optimizing, even one. Given your description, I’d guess that all
your segments will have deleted documents, so even if you do specify maxSegments on the optimize
command, the entire index will be rewritten.
        
        You’re in a bind, see: https://lucidworks.com/2017/10/13/segment-merging-deleted-documents-optimize-may-bad/.
You have this one massive segment and it will _not_ be merged until it’s almost all deleted
documents, see the link above for a fuller explanation.
        
        Prior to Solr 7.5 you don’t have many options except to re-index and _not_ optimize.
So if possible I’d reindex from scratch into a new collection and do not optimize. Or restructure
your process such that you can optimize in a quiet period when little indexing is going on.
        
        Best,
        Erick
        
        > On Jun 7, 2019, at 2:51 AM, jena <sthita2010@gmail.com> wrote:
        > 
        > Thanks @Nicolas Franck for reply, i don't see any any segment info for 4.4
        > version. Is there any API i can use to get my segment information ? Will try
        > to use maxSegments and see if it can help us during optimization.
        > 
        > 
        > 
        > --
        > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
        
        
    
Mime
View raw message