Santiago,
Thanks a lot for all your information. my comments in line
At 07:38 PM 3/21/2003 +0100, Santiago Gala wrote:
>Shan Gopalakrishnan wrote:
>>We observed few more things. I'm posting this to the best interest to
>>mature the framework further than what it is currently today.
>>Definitely this is not criticism.
>
>Feel free to criticize. It is the only way we have to improve Jetspeed.
--------->>Thanks and i will exchange whatever we learn part of our
analysis and testing
>>We are seeing the Garbage collector (Major GC) getting triggered
>>frequently as I think huge set of objects are getting created.
>
>Some profiling could be handy. AFAIK:
--------->> We will be doing this with OptimizeIt. will share that with you.
>Wrappers for security are not recycled (they *must* be inmutable)
>PortletSets are created new for every request, due to threading issues.
>--> Both kinds of objects should be claimed from "young" space, unless we
>are missing some "nullification" in permanent objects.
--------->>As we see the minor GC is not clearing up much objects, we still
think there is some missing nullification in the permanent objects
as we also notice the memory keeps growing.
>Every 30m you will have tomcat sessions expiring. If your testing does not
>keeps the sessions, you will be filling a lot of memory with fake sessions
>which will never be reused. I imagine you are using a testing framework
>that keeps sessions for a number of hits per "user".
--------->>we ensured this . the session id is passed through a cookie and
this is reused between requests from the same user
>A lot of other objects could come from either Velocity ASTs, Torque peers
>or other places. Even from your portlets ;-)
>
>About Spring 2001 some profiling and tuning was done, by the IBM team,
>David, Raphael and myself. I don't know of other such effort since then.
>This led to rundata and some other objects pooling in turbine, and some
>changes of String concatenation to StringBuffer.append(). We also
>discovered a lot of duplicated initializations and similar stuff. It is
>always funny to see how big bugs can hide in code.
--------->> This really makes us think twice whether we can use it for
production. I would appreciate if profiling/tuning can be treated much
frequently to develop a healthy framework. Hopefully some of our results
/analysis will help the development team.
>But the behaviour you are seeing looks more like the bug I pointed below
>or a bug in the hotspot compiler. Could you switch temporarily to jdk
>1.3.1_07 or 1.4.0_XX and see if the same happens? (According to reports in
>cocoon-dev, 1.4.0 does not have the StringBuffer bug)
--------->> I tried changing this. unfortunately changing it to 1.4.0.xx
resulted in a slower performance which is substantial and noticeable. i
couldn't believe. looked it and tried more than once to ensure it happens.
and yes it happens.
>>This could be inherited from Turbine implementation. We are now playing
>>with the various options in the Garbage Collector
>>part of the JDK 1.4.1, which is running the GC parallel, setting high
>>heap size, giving some % for young generation Vs old generation etc.
>>Most likely to my knowledge the minor GC looks only the younger
>>generation area as we didn't see any benefits in increase the
>>% of younger generation. Apparently the spike still remains and gives
>>poor response time just that the symptom happens way early
>>or later during the tests depending on the heap size and GC configuration.
>
>If the problem is that some "bunches" of objects get into "old" space
>because they are pointed by persistent objects, then a juditious ammount
>of "= null" on recycling can help a lot. I've just checked
>DefaultJetspeedRunData and it is disposed properly (unless something is
>broken down in Turbine and it is not really disposed).
----->>>Good point. I used to wonder how Jetspeed (team) will handle for
the inherent problems of Turbine ??
Thanks a lot for all your information.
- Shan
|