lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Indexing gets significantly slower after every batch commit
Date Fri, 22 May 2015 14:44:45 GMT
On 5/22/2015 3:15 AM, Angel Todorov wrote:
> Thanks for the feedback guys. What i am going to try now is deploying my
> SOLR server on a physical machine with more RAM, and checking out this
> scenario there. I have some suspicion it could well be a hypervisor issue,
> but let's see. Just for the record - I've noticed those issues on a Win
> 2008R2 VM with 8 GB of RAM and 2 cores.
> 
> I don't see anything strange in the logs. One thing that I need to change,
> though, is the verbosity of logs in the console - looks like by default
> SOLR outputs text in the log for every single document that's indexed, as
> well as for every query that's executed.

Bare metal will always perform better than a virtual machine.  Also,
Solr is *highly* threaded and really likes to have a lot of CPU cores.

Solr doesn't output a log line for every document indexed, unless you
are only including one document in each update request.  You should
definitely batch your updates -- put a few hundred or a few thousand of
them in each update request.  There is overhead to each request beyond
just the logging ... maximize the work done by each one.

I don't know that I would run Solr on Windows in production.  Windows
lags behind the free operating systems in memory management and
filesytem capabilities.  It's not that a Windows server is a BAD
environment, it's just that there are better ones that won't cost you money.

Thanks,
Shawn


Mime
View raw message