lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: tlogs not deleting
Date Tue, 19 Jun 2018 15:00:17 GMT
Take a look at the CDCR section of your reference guide, be sure you
get the version which you can download from here:
https://archive.apache.org/dist/lucene/solr/ref-guide/

There's the CDCR API call you can use for in-flight disabling, and
depending on the version of Solr you can set it in solrconfig.

Basically, buffering was there in the original CDCR to allow a larger
maintenance window, you could enable buffering and all updates were
saved until you disabled it, during which period you could do whatever
you needed with your target cluster and not lose any updates.

Later versions can do the full sync of the index and buffering is being removed.

Best,
Erick

On Tue, Jun 19, 2018 at 7:31 AM, Brian Yee <byee@wayfair.com> wrote:
> Thanks for the suggestion. Can you please elaborate a little bit about what DISABLEBUFFER
does? The documentation is not very detailed. Is this something that needs to be done manually
whenever this problem happens or is it something that we can do to fix it so it won't happen
again?
>
> -----Original Message-----
> From: Susheel Kumar [mailto:susheel2777@gmail.com]
> Sent: Monday, June 18, 2018 9:12 PM
> To: solr-user@lucene.apache.org
> Subject: Re: tlogs not deleting
>
> You may have to DISABLEBUFFER in source to get rid of tlogs.
>
> On Mon, Jun 18, 2018 at 6:13 PM, Brian Yee <byee@wayfair.com> wrote:
>
>> So I've read a bunch of stuff on hard/soft commits and tlogs. As I
>> understand, after a hard commit, solr is supposed to delete old tlogs
>> depending on the numRecordsToKeep and maxNumLogsToKeep values in the
>> autocommit settings in solrconfig.xml. I am occasionally seeing solr
>> fail to do this and the tlogs just build up over time and eventually
>> we run out of disk space on the VM and this causes problems for us.
>> This does not happen all the time, only sometimes. I currently have a
>> tlog directory that has 123G worth of tlogs. The last hard commit on
>> this node was 10 minutes ago but these tlogs date back to 3 days ago.
>>
>> We have sometimes found that restarting solr on the node will get it
>> to clean up the old tlogs, but we really want to find the root cause
>> and fix it if possible so we don't keep getting disk space alerts and
>> have to adhoc restart nodes. Has anyone seen an issue like this before?
>>
>> My update handler settings look like this:
>>   <updateHandler class="solr.DirectUpdateHandler2">
>>
>>       <updateLog class="solr.CdcrUpdateLog">
>>
>>       <str name="dir">${solr.ulog.dir:}</str>
>>       <int name="numVersionBuckets">${solr.ulog.numVersionBuckets:
>> 65536}</int>
>>     </updateLog>
>>     <autoCommit>
>>         <maxTime>600000</maxTime>
>>         <maxDocs>250000</maxDocs>
>>         <openSearcher>false</openSearcher>
>>     </autoCommit>
>>     <autoSoftCommit>
>>         <maxTime>120000</maxTime>
>>     </autoSoftCommit>
>>
>>   <shardHandlerFactory class="HttpShardHandlerFactory">
>>     <int name="maxConnectionsPerHost">100</int>
>>   </shardHandlerFactory>
>>
>>   </updateHandler>
>>

Mime
View raw message