lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <elyog...@elyograg.org>
Subject Re: upgrading 1hr autoCommit behavior
Date Tue, 11 Jun 2013 21:58:14 GMT
On 6/11/2013 3:45 PM, Michael Tsadikov wrote:
> We're upgrading our Solr 3.1 distributed masters/slaves setup to 4.3.
>
> In 3.1 we used autoCommit every hour on masters, each commit is replicated
> to slaves, and all searches are done on slaves. 1hr visibility is ok - we
> don't need NRT.
>
> In 4.3 we enable transaction logs, for durability in case of crashes.
>
> However, we are wondering how to best setup autoCommit/autoSoftCommit on
> masters to preserve the old behavior. It seems that setting autoCommit to
> 1hr (openSearcher=true) without any autoSoftCommit preserves our previous
> setup - is this correct? Wil the transaction log make masters use much more
> heap due to 1hr periods between commits? This can be a problem for us
> because we put many master cores on one solr JVM

If you want to completely preserve your previous setup, then you've got 
it correct.  Depending on how much you index over the course of that 
hour, you might want to go a different way.

If you are doing hard commits once an hour, you'll end up with at least 
2 hours of indexing in your transaction log.  If a few hours of indexing 
represents a lot of data, then it can take a really long time to replay 
those transactions when Solr restarts.  If it's not a lot of data, then 
you shouldn't have any trouble.

http://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup

If a few hours of indexing data *IS* a lot, then autoCommit every five 
minutes with openSearcher=false and autoSoftCommit once an hour would 
preserve your current visibility characteristics without giving you 
really huge transaction logs.

Thanks,
Shawn


Mime
View raw message