lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <s...@elyograg.org>
Subject Re: SOLR memory usage jump in JVM
Date Wed, 19 Sep 2012 12:20:41 GMT
On 9/18/2012 9:29 PM, Lance Norskog wrote:
> There is a known JVM garbage collection bug that causes this. It has to do with reclaiming
Weak references, I think in WeakHashMap. Concurrent garbage collection collides with this
bug and the result is that old field cache data is retained after closing the index. The bug
is more common with more processors doing GC simultaneously.
>
> The symptom is that when you run a monitor, the memory usage rises to a peak, drops to
a floor, rises again in the classic sawtooth pattern. When the GC bug happens, the ceiling
becomes the floor, and the sawtooth goes from the new floor to a new ceiling. The two sizes
are the same. So, 2G to 5G, over and over, suddenly it is 5G to 8G, over and over.
>
> The bug is fixed in recent Java 7 releases. I'm sorry, but I cannot find the bug number.

I think I ran into this when I was looking at memory usage on my SolrJ 
indexing program.  Under Java6, memory usage in jconsole (remotely via 
JMX) was fairly constant long-term (aside from the unavoidable 
sawtooth).  When I ran it under Java 7u3, it would continually grow, 
slowly ... but if I measured it with jstat on the Linux commandline 
rather than remotely via jconsole under windows, memory usage was 
consistent over time, just like under java6 with the remote jconsole.  
After looking at heap dumps and scratching my head a lot, I finally 
concluded that I did not have a memory leak, there was a problem with 
remote JMX monitoring in java7.  Glad to hear I was not imagining it, 
and that it's fixed now.

Thanks,
Shawn


Mime
View raw message