cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joerg Heinicke <joerg.heini...@gmx.de>
Subject Re: Javaflow - major memory issue
Date Wed, 02 Apr 2008 02:54:13 GMT
On 01.04.2008 18:52, footh wrote:

> Just ran a very simple test using the changes from revision 642694 and had unusual results.
> 
> Byte arrays, and consequently the BufferedOutputStream still took up large amounts of
retained
> space in the total memory usage.  So, this didn't change.
> 
> However, the memory did not clean up neatly like it did before...after about 10 minutes.
 Before,
> I would do, say 1000 samples, and total Tomcat memory would balloon to a point.  Then
I'd wait 10
> minutes and would observe the byte array data cleaning up in the profiler with total
Tomcat memory
> staying the same.  After that, I'd run another 1000 samples and the total Tomcat memory
would not
> increase again until hitting around the 1000th sample and the cycle would repeat.
> 
> With these new changes, the memory cleaned up a little bit, but not nearly as much as
before.  And
> the total memory would start increasing before I hit the control sample size.
> 
> The BufferedOutputStream path appears to be the same as well - flowing up to
> ContinuationsManagerImpl.  Perhaps I'm doing something wrong?  I did make this changes
to version
> 2.1.10 NOT 2.1.11 since I haven't upgraded my site to the new version yet.

That's no good news :( And I can't see how this should be possible.

Which changes exactly did you apply? Only rev 642694 [1]? The actual two 
files changed are ContinuationContext (added method onSuspend()) and 
AbstractContinuable (calling context.onSuspend() right before 
Continuation.suspend()). The method only nulls out some values, so I 
can't see how this should have worsen the situation. Both files had not 
changed between 2.1.10 and 2.1.11, so this can neither be a reason.

Second, this was only the first approach. After Torsten's review I did a 
more "brave" approach setting the complete context to null before the 
continuation gets suspended [2]. Here I reverted the above 2 changes to 
AbstractContiuable and ContinuationsContext. The fix went into 
Continuation (storing functionName and setting context to null in 
suspend()) and JavaInterpreter (instantiating Continuation with 
functionName and retrieving it from there later on). Again I can't see 
how this should have worsen the situation. Both files had again not 
changed since 2.1.10.

Then in order to fix COCOON-2109 [3] I applied another fix [4]. If that 
one helps you at all pretty much depends on whether you access old 
continuations or not (when using a wizard you not only go forward along 
the expected path (page 1, page 2, etc.) but also go back).

The last important fix is for synchronization issues in 
ContinuationsManagerImpl [5]. This definitely has some impact, it's 
usually unlikely that fixing synchronization improves performance. If 
you don't have this fix I can imagine that the situation is as bad as 
you describe it for the following reasons: If something creates a new 
continuation while the clean up thread for removing expired 
continuations is active it kicks the latter one with a 
ConcurrentModificationException - and the clean up just stops. This 
would mean though that it was just bad luck in your new test.

I don't know how exactly you profiled your system. You should definitely 
NOT run on debug log level. I fixed the synchronization issues in favor 
of the debug log level: There are no longer two sets maintained but the 
second set (which was for debugging purposes only) gets recreated "on 
demand", i.e. debug log level, which locks the manager for that time. 
But it should improve the overall situation though.

Joerg

[1] http://svn.apache.org/viewvc?view=rev&revision=642694
[2] http://svn.apache.org/viewvc?view=rev&revision=642756
[3] https://issues.apache.org/jira/browse/COCOON-2109
[4] http://svn.apache.org/viewvc?view=rev&revision=642843
[5] http://svn.apache.org/viewvc?view=rev&revision=643295

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org


Mime
View raw message