lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: Solr 4.2 Startup Detects Corrupt Log And is Really Slow to Start
Date Wed, 17 Apr 2013 17:42:02 GMT
On 4/17/2013 10:29 AM, Umesh Prasad wrote:
> We use DIH and have turned off the Auto commit because we have to sometimes
> build index from Scratch (clean=true) and we not want to
> Our master server sees a lot of restarts, sometimes 2-3 times a day. It
> polls other Data Sources for updates which are quite a few. Master
> maintains a version of last committed version and can handle uncommitted
> changes.
>
> Given the frequent restarts, We can't really afford a huge start up at this
> point.
>   In the worst case, does Solr allow for disabling transactional log ?

Unless you are using SolrCloud, you can disable the updateLog. 
SolrCloud requires it.

There is one additional caveat - when you disable the updateLog, you 
have to switch to MMapDirectoryFactory instead of 
NRTCachingDirectoryFactory.  The NRT directory implementation will cache 
a portion of a commit (including hard commits) into RAM instead of onto 
disk.  On the next commit, the previous one is persisted completely to 
disk.  Without a transaction log, you can lose data.

My advice - keep the updateLog on, and use autoCommit with 
openSearcher=false.  It is the best way to avoid large transaction logs. 
  It sounds like you do not want the auto commits to affect query 
results, which is a reasonable goal.  You can have that even with 
autoCommit - just set openSearcher to false.  Here's an example, no need 
to stick with the numbers that I have included:

<updateHandler class="solr.DirectUpdateHandler2">
   <autoCommit>
     <maxDocs>25000</maxDocs>
     <maxTime>300000</maxTime>
     <openSearcher>false</openSearcher>
   </autoCommit>
   <updateLog />
</updateHandler>

Thanks,
Shawn


Mime
View raw message