lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Expunge deleting using excessive transient disk space
Date Sun, 08 Sep 2013 18:30:15 GMT
Right, but you should have at least as much free space as your total index
size, and I don't see the total index size (but I'm just glancing).

I'm not entirely sure you can precisely calculate the maximum free space
you have relative to the amount needed for merging, some of the people who
wrote that code can probably tell you more.

I'd _really_ try to get more disk space. The amount of engineer time spent
trying to tune this is waaaaay more expensive than a disk...

Best,
Erick


On Sun, Sep 8, 2013 at 11:51 AM, Manuel Le Normand <
manuel.lenormand@gmail.com> wrote:

>   Hi,
> In order to delete part of my index I run a delete by query that intends to
> erase 15% of the docs.
> I added this params to the solrconfig.xml
> <mergePolicy class="org.apache.lucene.index.TieredMergePolicy">
>    <int name="maxMergeAtOnce">2</int>
>    <int name="maxMergeAtOnceExplicit">2</int>
>    <double name="maxMergedSegmentMB">5000.0</double>
>    <double name="reclaimDeletesWeight">10.0</double>
>    <double name="segmentsPerTier">15.0</double>
> </mergePolicy>
>
> The extra params were added in order to promote merge of old segments but
> with restriction on the transient disk that can be used (as I have only
> 15GB per shard).
>
> This procedure failed on a no space left on device exception, although
> proper calculations show that these params should cause no usage excess of
> the transient free disk space I have.
>  Looking on the infostream I can see that the first merges do succeed but
> older segments are kept in reference thus cannot be deleted until all the
> merging are done.
>
> Is there anyway of overcoming this?
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message