forrest-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thorsten Scherler <thorsten.scherler....@juntadeandalucia.es>
Subject Re: memory leaks in forrest
Date Thu, 31 Jul 2008 07:01:44 GMT
On Thu, 2008-07-31 at 12:34 +1000, David Crossley wrote:
> Thorsten Scherler wrote:
> > 
> > lately I saw a couple of OutOfMemoryError reports around forrest. Brain
> > and Gavin reported them around the dispatcher plugin. Actually today I
> > ran to an OutOfMemoryError within our solr plugin. 
> > 
> > The code is not used with the dispatcher but directly. I am using 
> > http://svn.apache.org/viewvc/forrest/trunk/whiteboard/plugins/org.apache.forrest.plugin.output.solr/src/java/org/apache/forrest/solr/client/SolrSearchGenerator.java?view=log
> > 
> > Doing a "top" on the box I see that the memory never got cleaned up. It
> > keeps on growing till it reaches the max. You can see it quite fast in
> > limiting the max memory to a real small amount (e.g. 128M) the memory
> > usage is keep on growing till forrest fails with Heaps size error. 
> > 
> > My first impression would be that is caused by the above generator but
> > seeing all the recent threads about similar situations it makes me
> > wander whether that may be a general problem.
> 
> I don't use the solr plugin.
> 
> I do see the issue with one of my complex sites that
> uses dispatcher plugins (not your other whiteboard work).
> Also with the forrest/site-author site which is skins-based.
> 
> The memory usage does not increase quickly, as for yours,
> but yes it does increase. Only manual tests so not yet
> reached a crash.
> 
> However, this seems to be consistent with the bugs that
> we have already identified.
> https://issues.apache.org/jira/browse/FOR-591
> "Memory Leak with XMLFileModule"
> and there was some other known dispatcher issues.
> 

Thank you very much David to point me to this issue. I will now
investigate whether this can be the cause (but it really sounds like it)
and see how we can fix this. 


> Aside: Oh it would be blissful to get our internal
> Cocoon upgraded to at least use Cocoon-2.1 head.
> Ralph Goers has fixed that problem which causes FOR-591.

I first need to find a quick workaround for the project based on
forrest in my work (need to deliver till Monday a new version of this
work project), but then I need to fix this issue once and for all in
forrest and will try to update our cocoon. I personally prefer to 2.2
but I fear the pass-through feature (missing in 2.2. AFAIK and
extensively used in forrest) will be a showstopper but 2.1 should work
just fine.

> 
> > Does anybody see an obvious leak in the above generator? Does anybody
> > has an idea how we can trigger the leak?
> 
> I have not looked - perhaps someone else can.
> 
> Does the solr plugin make extensive use of XML File Module?
> You might reach crisis more quickly.
> 
> Also see Tim's comments in FOR-591 regarding perhaps
> "compounded this issue with similar simple caching in the locationmap".
> Maybe that part has already been addressed.
> There was recently some other locationmap changes.
> 
> Dunno, sorry, just some ideas.

Thank you very much for your thoughts. As usual most valuable.

salu2

> 
> -David
-- 
Thorsten Scherler                                 thorsten.at.apache.org
Open Source Java                      consulting, training and solutions


Mime
View raw message