incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Hirsch <hirsch.d...@gmail.com>
Subject Re: Memory allocation analysis Episode II
Date Sat, 28 Nov 2009 08:40:22 GMT
Excellent idea.

Let me know if I can help.

I also think this public discussion regarding lift/scala performance
is critical for the acceptance of both in the enterprise.

D.



On Sat, Nov 28, 2009 at 9:00 AM, Markus Kohler <markus.kohler@gmail.com> wrote:
> Hi Richard,
> taking this to the mailing list.
> My plan was always to promote ESME to developers by providing them some
> valuable information about how to optimize a real world Scala/Lift
> application. If we promote the blog post a bit through dzone, reddit etc. we
> should be able to get a few thousand page views on a single day. To be
> valuable I would have to do a little bit more work, e.g. numbers for the
> response time before and after would certainly something people would ask
> for. It would also good to clarify the Scala loadXML issue, something I
> wanted to do anyway soon.
>  A little introduction into memory allocation tracing would also make
> sense.
>  I could work on this on Monday night, if nothing else is more important.
> Any comments here?
>
> Regards,
> Markus
>
> "The best way to predict the future is to invent it" -- Alan Kay
>
>
> On Sat, Nov 28, 2009 at 7:10 AM, Richard Hirsch <hirsch.dick@gmail.com>wrote:
>
>> The bar chart is a great idea. Would be perfect for a blog. We can
>> show we had a memory prob with a reference to a particular wiki page.
>> Then describe the code change and then put a link to another wiki
>> page. If you create the chart and do a very quick write-up, just
>> blurbs. I'll blend everything and write the blog.
>>
>> Thanks.
>>
>> D.
>>
>> On Sat, Nov 28, 2009 at 12:30 AM, Markus Kohler <markus.kohler@gmail.com>
>> wrote:
>> > Hi,
>> > -ESME out of svn 27.11.2009 running on jetty default setting. Turned out
>> to
>> > be only 64 Mbyte
>> > - logged all 300+x users
>> > - GC graph 1. attachment yellow line indicates 323 users logged on only
>> need
>> > around 52Mbyte
>> > -Still GC issues when sending one message, but not as bad as before
>> > Restarted jetty and did a memory allocation tracing profiling session for
>> > one message send; details see 2. attachment
>> > -Overall 18.6 Mbyte allocated
>> > - 9,9 Mbyte in scala.xml.XML$.loadString(java.lang.String) >50% overall,
>> see
>> > 3. attachment.
>> >
>> > Maybe we should just start to provide a simple bar chart.
>> > eg. we went down from 90-100 to 18,6.
>> > I would like to do a cpu profiling session for one message send, but ATM
>> > have problems with my network. Intend to post the result to the scala dev
>> > list. Maybe there's a workaround and we can use some other Class/method.
>> >
>> > Greetings,
>> > Markus
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > "The best way to predict the future is to invent it" -- Alan Kay
>> >
>> >
>> > On Fri, Nov 27, 2009 at 10:29 PM, Richard Hirsch <hirsch.dick@gmail.com>
>> > wrote:
>> >>
>> >> @David - I don't know whether the Scala community is following these
>> >> threads. Maybe, you can point them to Markus' work and ask them to
>> >> take a look at the memory usage of "scala.xml.XML$.loadString".
>> >>
>> >> @Markus if you'll sen me the reports I'll post another wiki entry and
>> >> ten blog about it.
>> >>
>> >>
>> >> Thanks.
>> >>
>> >> D.
>> >>
>> >>
>> >> On Fri, Nov 27, 2009 at 5:28 PM, Markus Kohler <markus.kohler@gmail.com
>> >
>> >> wrote:
>> >> > Hi all,
>> >> > In short, We are down from 90Mbyte to *19Mbyte* for sending one
>> message
>> >> > and
>> >> > I believe it should be possible to get under 10Mbyte  without too
much
>> >> > effort. With 10  Mbyte we would comply to the performance standard
of
>> >> > some
>> >> > major ERP vendor, but honestly I believe we should be able to do even
>> >> > better
>> >> > ;-)
>> >> >
>> >> > Out of the 19Mbyte(maybe even only 11, it's bit hard to get
>> reproducible
>> >> > results) 9,9 are consumed by
>> scala.xml.XML$.loadString(java.lang.String)
>> >> > mainly because every time the routine is called, a new Parser is
>> >> > instantiated. In addition
>> >> > javax.xml.parsers.SAXParserFactory.newInstance()
>> >> > is called, which hits  (unless that has changed in recent JDK's) the
>> >> > file
>> >> > system to find out which XML parser is configured.  *I think ideally
>> >> > this
>> >> > should be fixed in the Scala sources. *
>> >> >
>> >> > The issue with the message formatted 300 times when all users are
>> logged
>> >> > on
>> >> > still seems to be there, at least the GC log suggests that, but I
>> guess
>> >> > David will attack that soon. I will send around the profiling reports
>> >> > later
>> >> > on.
>> >> >
>> >> > Regards,
>> >> > Markus
>> >> >
>> >> >
>> >> > "The best way to predict the future is to invent it" -- Alan Kay
>> >> >
>> >
>> >
>>
>

Mime
View raw message