lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Frequent softCommits leading to high faceting times?
Date Mon, 16 Sep 2013 12:56:31 GMT
Soft commits are not free, they invalidate certain caches
which then have to be reloaded. I suspect you're hitting
this big time. The question is always "do you really, really
_need_ 1 second latency?". Set the soft commit interval to
be as long as your application can stand IMO.

And it may have nothing to do with facets, since things
like your filterCache autowarming is done on soft commit.
Ditto for the other caches you've configured in solrconfig.xml.

But one thing to watch is the size of your tlog. Transaction
logs are only truncated on hard commits, and can get
replayed on restart. So you're risking long restart times here.

See:
http://searchhub.org/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

FWIW,
Erick


On Mon, Sep 16, 2013 at 1:43 AM, Rohit Kumar <rohit.kkumar@gmail.com> wrote:

> Hi,
>
> We are running *SOLR 4.3* with 8 Gb of index on
>
> Ubuntu 12.04 64 bits
> Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz Single core.
> 16GB RAM
>
>
> We just started using the autoSoftCommit feature and noticed the facet
> queries slowed down from milliseconds taking earlier to a minute. We have
> *8
> facet fields*.
>
> We add close to 300 documents per second during peak interval.
>
> <autoCommit>
>     <maxTime>600000</maxTime>
>     <openSearcher>false</openSearcher>
> </autoCommit>
>
> <autoSoftCommit>
>     <maxTime>1000</maxTime>
> </autoSoftCommit>
>
>
> Here is some information i got with debugQuery. Please note that *facet
> time is more than 50 seconds.*
>
> <lst name="process">
>     <double name="time">50779.0</double>
>     <lst name="network">
>         <double name="time">0.0</double>
>     </lst>
>     <lst name="query">
>         <double name="time">41.0</double>
>     </lst>
>     *<lst name="facet">
>         <double name="time">50590.0</double>
>     </lst>*
>     <lst name="mlt">
>         <double name="time">0.0</double>
>     </lst>
>     <lst name="highlight">
>         <double name="time">0.0</double>
>     </lst>
>     <lst name="stats">
>         <double name="time">0.0</double>
>     </lst>
>     <lst name="connection">
>         <double name="time">5.0</double>
>     </lst>
>     <lst name="debug">
>         <double name="time">143.0</double>
>     </lst>
> </lst>
>
> Please help.
>
> Thanks,
> Rohit Kumar
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message