cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simone Gianni <s.gia...@thebug.it>
Subject Continuations consume ram, possible solutions?
Date Wed, 26 Jul 2006 17:31:40 GMT
Hi Ard,
switching to dev, I also think it's more appropiate.

Unfortunately I didn't have time in the last period to work a lot on
cocoon issues, but I hope I will have some during this summer.

IMMO the main problem is not with "middle" continuations, but it's
actually with first continuations. First continuations are the root of
the continuation tree, and it happens often that they are created and
never really used, for example because they are the just a box aside the
main site pages, or because they are the first step of a registration or
something else that is presented to the a user that leaves after the
first page. Do you agree with me that also in your situation great part
of continuations are actually "first continuations"?

IIRC we talked about three solutions:
- Disk store for continuation
- Named continuations
- Light-continuations for entry pages

The first one is quite obvious to explain, not so simple to implement
and does not always solve the problem. While I think is rather simple to
serialize to disk (thru EHCache or anything else) all the native
javascript/java variables, it's a problem to do it on objects : even if
your business objects implements Serializable, that does not mean that
they can be safely serialized and deserialized, and often serializing
them does not necessarily mean you are freeing ram.

For example if you are using hibernate persistence, reference to your
loaded objects are kept in the hibernate session, and something similar
happens with jpox too and so on.

Anyway, we could simply leave this problem as a programmer's concern,
and provide a facility to store and retrieve from the disk
continuations, only if all objects contained in it are serializable.
Unfotunately I don't think it will be simple to have serializable cocoon
forms anyway.

The second solution is very wise, and IIRC you already have something
implemented. Using named continuations mean that every continuation have
a "name", and there can only one continuation tree with a given name in
the http session. This way, if i have a certain search page that
everybody uses, when the user makes a search the continuation is created
and eventually its subcontinuations, but when the user goeas away from
the page and then returns to the search page, a new continuation with
the same name is created, which will replace the old tree, so freeing
the ram. This way, if a typical site pattern is to do 10 searches,
instead of having 10 continuation trees in RAM until they expire, we
have only one.

This would not only solve "first continuations" but free the entire
continuation trees. We could ease it a bit using the entry function name
as the continuation name, and having a "singleton" notation somewhere
(cocoon.setSingletonContinuations()) to mark those scripts that uses
single named continuations.

The third option aims to solve the common situation of having a flow
that initializes some variables, sends a form or a page (creating a
continuation) and then waits for the user to click on a button, that 90%
of time never gets pushed. This is quite common in aggregated pages with
boxes or something similar.

The idea here is NOT saving anything in the continuation, instead when
the continuation is recalled, the function is executed again from the
beginning, but skipping the first sendPage (since that page has already
been sent). This way we don't have any serialization issue, the form is
created, displayed and then garbage collected, and recreated when the
user clicks the button, not displayed again since the first sendPage is
skipped, but only populated from the request.

This is maybe the simplest one to implement, but is quite untidy because
if the code in the first lines execute something heavy or some business
logic, it could not be clear why it's getting executed twice. But since
it's aimed to solve the problem of simple forms generated many times
(like login boxes, polls, subscribe here boxes and so on), it's highly
possible that the first lines of the flow are simply instantiating a
bean or a document, creating a form, doing some binding and displaying
it, and if it's really a box aside the pages of the side that code is
already executed at every get.

Do you (plural, rest of cocoon community) have any other idea about
this? "Continuation pollution" is actually a problem in flow
implementations.

Simone


Ard Schrijvers wrote:

>Hello Simone,
>
>talking about continuations, did you already find a solid way to handle high traffic sites
with many concurrent continuations and memory useage? It bothers me a little that when building
sites, we have to keep track of the number of continuations (we had a large site with many
visitors and a poll on the homepage having a continuation. This brought the site down a few
times).
>
>You mentioned serializing continuations to diskStore (ofcourse, the flowscript/javaflow
writer should make sure al things in the cont are serializable then). Is this feasible?
>
>The other day I also thought about the cocoon caches having this StoreJanitor trying to
free memory from it when JVM is low on memory. It just does not make sense to me, that this
is only tried regarding the cache, while currently, also continuations might be the reason
for a low on memory JVM. 
>
>Suppose, I have a healthy cache, nicely populated, and a high cachehit range, but it happened
to be that many many continuations have been created, all long lived (5 hours), and all quite
large (1 continuation can be very large in memory). Now, due to this continuations, JMV is
low on memory, implying the StoreJanitor to run, removing my cache, and certainly not solving
any problem.
>
>So I was wondering if you had some new ideas on this subject...though, perhaps the dev-list
is more appropriate for it.
>
>WDYT?
>
>Regards Ard  
>
>  
>
>>Hi Toby,
>>I think you are right. What a continuation does (should do) 
>>is dump the
>>local variables and restore them before restarting the flow. 
>>This means
>>that if you write var a = 1; then create a continuation, when 
>>you return
>>to that continuation a should be 1 again, even if in other 
>>continuations
>>it has been changed to 2,3 or 4.
>>
>>There is surely one known limitation to this : if you say var 
>>bean = new
>>MyBean(); bean.setX(1); then produce a continuation, then after the
>>continuation you call bean.setX(2), even if you go back to 
>>the previous
>>continuation you will find that bean.getX() == 2, because the LOCAL
>>VARIABLE is still your bean, but it's internal state is not manageable
>>by the continuation (more formally, your local variable is a 
>>pointer to
>>a bean, which is correctly restored when you go back to the
>>continuation, but the data it points to is not serialized/deserialized
>>by the continuation).
>>
>>But this is not your case, in this case you are setting a simple
>>javascript variable, so it should work as you say, at least AFAIK :)
>>
>>Please, file a bug about it.
>>
>>Simone
>>
>>Toby wrote:
>>
>>    
>>
>>>Jason Johnston wrote:
>>> 
>>>
>>>      
>>>
>>>>First you assign the 'useless' variable a String value, then you
>>>>create the continuation.  When you resume the continuation the first
>>>>time, you re-assign the 'useless' variable so that it now holds an
>>>>Array value (String.split() returns an Array).  When you resume the
>>>>continuation again, you try to call .split() on the 'useless' var,
>>>>which is now an Array, and the error is appropriately 
>>>>        
>>>>
>>thrown since an
>>    
>>
>>>>Array has no such method.
>>>>   
>>>>
>>>>        
>>>>
>>>When I resume the continuation again, I'm resuming it from before
>>>assigning the array, so one would think that useless (which 
>>>      
>>>
>>is a local
>>    
>>
>>>variable) would still contain a string!
>>>
>>>What am I missing?
>>>
>>>
>>>Toby
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
>>>For additional commands, e-mail: users-help@cocoon.apache.org
>>>
>>> 
>>>
>>>      
>>>
>>-- 
>>Simone Gianni
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
>>For additional commands, e-mail: users-help@cocoon.apache.org
>>
>>
>>    
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
>For additional commands, e-mail: users-help@cocoon.apache.org
>
>  
>
-- 
Simone Gianni

Mime
View raw message