lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Yee <>
Subject RE: tlogs not deleting
Date Tue, 19 Jun 2018 15:50:02 GMT
Thank you Erick. I am running Solr 6.6. From the documentation:
"Replicas do not need to buffer updates, and it is recommended to disable buffer on the target

Do you recommend disabling the buffer on the source SolrCloud as well? It looks like I already
have the buffer disabled at target locations but not the source location. Would it even make
sense at the source location?

This is what I have at the target locations:
<requestHandler name="/cdcr" class="solr.CdcrRequestHandler">
  <shardHandlerFactory class="HttpShardHandlerFactory">
      <int name="maxConnectionsPerHost">100</int>
  <lst name="buffer">
    <str name="defaultState">disabled</str>

-----Original Message-----
From: Erick Erickson [] 
Sent: Tuesday, June 19, 2018 11:00 AM
To: solr-user <>
Subject: Re: tlogs not deleting

Take a look at the CDCR section of your reference guide, be sure you get the version which
you can download from here:

There's the CDCR API call you can use for in-flight disabling, and depending on the version
of Solr you can set it in solrconfig.

Basically, buffering was there in the original CDCR to allow a larger maintenance window,
you could enable buffering and all updates were saved until you disabled it, during which
period you could do whatever you needed with your target cluster and not lose any updates.

Later versions can do the full sync of the index and buffering is being removed.


On Tue, Jun 19, 2018 at 7:31 AM, Brian Yee <> wrote:
> Thanks for the suggestion. Can you please elaborate a little bit about what DISABLEBUFFER
does? The documentation is not very detailed. Is this something that needs to be done manually
whenever this problem happens or is it something that we can do to fix it so it won't happen
> -----Original Message-----
> From: Susheel Kumar []
> Sent: Monday, June 18, 2018 9:12 PM
> To:
> Subject: Re: tlogs not deleting
> You may have to DISABLEBUFFER in source to get rid of tlogs.
> On Mon, Jun 18, 2018 at 6:13 PM, Brian Yee <> wrote:
>> So I've read a bunch of stuff on hard/soft commits and tlogs. As I 
>> understand, after a hard commit, solr is supposed to delete old tlogs 
>> depending on the numRecordsToKeep and maxNumLogsToKeep values in the 
>> autocommit settings in solrconfig.xml. I am occasionally seeing solr 
>> fail to do this and the tlogs just build up over time and eventually 
>> we run out of disk space on the VM and this causes problems for us.
>> This does not happen all the time, only sometimes. I currently have a 
>> tlog directory that has 123G worth of tlogs. The last hard commit on 
>> this node was 10 minutes ago but these tlogs date back to 3 days ago.
>> We have sometimes found that restarting solr on the node will get it 
>> to clean up the old tlogs, but we really want to find the root cause 
>> and fix it if possible so we don't keep getting disk space alerts and 
>> have to adhoc restart nodes. Has anyone seen an issue like this before?
>> My update handler settings look like this:
>>   <updateHandler class="solr.DirectUpdateHandler2">
>>       <updateLog class="solr.CdcrUpdateLog">
>>       <str name="dir">${solr.ulog.dir:}</str>
>>       <int name="numVersionBuckets">${solr.ulog.numVersionBuckets:
>> 65536}</int>
>>     </updateLog>
>>     <autoCommit>
>>         <maxTime>600000</maxTime>
>>         <maxDocs>250000</maxDocs>
>>         <openSearcher>false</openSearcher>
>>     </autoCommit>
>>     <autoSoftCommit>
>>         <maxTime>120000</maxTime>
>>     </autoSoftCommit>
>>   <shardHandlerFactory class="HttpShardHandlerFactory">
>>     <int name="maxConnectionsPerHost">100</int>
>>   </shardHandlerFactory>
>>   </updateHandler>
View raw message