lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Solr Memory Usage
Date Wed, 29 Oct 2014 23:00:08 GMT
On 10/29/2014 1:05 PM, Toke Eskildsen wrote:
> We did have some problems on a 256GB machine churning terabytes of data through 40 concurrent
Tika processes and into Solr. After some days, performance got really bad. When we did a top,
we noticed that most of the time was used in the kernel (the 'sy' on the '%Cpu(s):'-line).
The drop_caches trick worked for us too. Our systems guys explained that it was because of
virtual memory space fragmentation, so the OS had to spend a lot of resources just bookkeeping
memory.

There's always at least one exception to any general advice, including
whatever I come up with!  It's really too bad that it didn't Just Work
(tm) for you.  Weird things can happen when you start down the path of
extreme scaling, though.

Thank you for exploring the bleeding edge for us!

Shawn


Mime
View raw message