esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Hirsch <hirsch.d...@gmail.com>
Subject Re: Memory allocation analysis Episode II
Date Sat, 28 Nov 2009 10:56:41 GMT
Added performance test results from 11-27 to the wiki:
http://cwiki.apache.org/ESME/performance-tests.html

On Sat, Nov 28, 2009 at 9:40 AM, Richard Hirsch <hirsch.dick@gmail.com> wrote:
> Excellent idea.
>
> Let me know if I can help.
>
> I also think this public discussion regarding lift/scala performance
> is critical for the acceptance of both in the enterprise.
>
> D.
>
>
>
> On Sat, Nov 28, 2009 at 9:00 AM, Markus Kohler <markus.kohler@gmail.com> wrote:
>> Hi Richard,
>> taking this to the mailing list.
>> My plan was always to promote ESME to developers by providing them some
>> valuable information about how to optimize a real world Scala/Lift
>> application. If we promote the blog post a bit through dzone, reddit etc. we
>> should be able to get a few thousand page views on a single day. To be
>> valuable I would have to do a little bit more work, e.g. numbers for the
>> response time before and after would certainly something people would ask
>> for. It would also good to clarify the Scala loadXML issue, something I
>> wanted to do anyway soon.
>>  A little introduction into memory allocation tracing would also make
>> sense.
>>  I could work on this on Monday night, if nothing else is more important.
>> Any comments here?
>>
>> Regards,
>> Markus
>>
>> "The best way to predict the future is to invent it" -- Alan Kay
>>
>>
>> On Sat, Nov 28, 2009 at 7:10 AM, Richard Hirsch <hirsch.dick@gmail.com>wrote:
>>
>>> The bar chart is a great idea. Would be perfect for a blog. We can
>>> show we had a memory prob with a reference to a particular wiki page.
>>> Then describe the code change and then put a link to another wiki
>>> page. If you create the chart and do a very quick write-up, just
>>> blurbs. I'll blend everything and write the blog.
>>>
>>> Thanks.
>>>
>>> D.
>>>
>>> On Sat, Nov 28, 2009 at 12:30 AM, Markus Kohler <markus.kohler@gmail.com>
>>> wrote:
>>> > Hi,
>>> > -ESME out of svn 27.11.2009 running on jetty default setting. Turned out
>>> to
>>> > be only 64 Mbyte
>>> > - logged all 300+x users
>>> > - GC graph 1. attachment yellow line indicates 323 users logged on only
>>> need
>>> > around 52Mbyte
>>> > -Still GC issues when sending one message, but not as bad as before
>>> > Restarted jetty and did a memory allocation tracing profiling session for
>>> > one message send; details see 2. attachment
>>> > -Overall 18.6 Mbyte allocated
>>> > - 9,9 Mbyte in scala.xml.XML$.loadString(java.lang.String) >50% overall,
>>> see
>>> > 3. attachment.
>>> >
>>> > Maybe we should just start to provide a simple bar chart.
>>> > eg. we went down from 90-100 to 18,6.
>>> > I would like to do a cpu profiling session for one message send, but ATM
>>> > have problems with my network. Intend to post the result to the scala dev
>>> > list. Maybe there's a workaround and we can use some other Class/method.
>>> >
>>> > Greetings,
>>> > Markus
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > "The best way to predict the future is to invent it" -- Alan Kay
>>> >
>>> >
>>> > On Fri, Nov 27, 2009 at 10:29 PM, Richard Hirsch <hirsch.dick@gmail.com>
>>> > wrote:
>>> >>
>>> >> @David - I don't know whether the Scala community is following these
>>> >> threads. Maybe, you can point them to Markus' work and ask them to
>>> >> take a look at the memory usage of "scala.xml.XML$.loadString".
>>> >>
>>> >> @Markus if you'll sen me the reports I'll post another wiki entry and
>>> >> ten blog about it.
>>> >>
>>> >>
>>> >> Thanks.
>>> >>
>>> >> D.
>>> >>
>>> >>
>>> >> On Fri, Nov 27, 2009 at 5:28 PM, Markus Kohler <markus.kohler@gmail.com
>>> >
>>> >> wrote:
>>> >> > Hi all,
>>> >> > In short, We are down from 90Mbyte to *19Mbyte* for sending one
>>> message
>>> >> > and
>>> >> > I believe it should be possible to get under 10Mbyte  without
too much
>>> >> > effort. With 10  Mbyte we would comply to the performance standard
of
>>> >> > some
>>> >> > major ERP vendor, but honestly I believe we should be able to do
even
>>> >> > better
>>> >> > ;-)
>>> >> >
>>> >> > Out of the 19Mbyte(maybe even only 11, it's bit hard to get
>>> reproducible
>>> >> > results) 9,9 are consumed by
>>> scala.xml.XML$.loadString(java.lang.String)
>>> >> > mainly because every time the routine is called, a new Parser is
>>> >> > instantiated. In addition
>>> >> > javax.xml.parsers.SAXParserFactory.newInstance()
>>> >> > is called, which hits  (unless that has changed in recent JDK's)
the
>>> >> > file
>>> >> > system to find out which XML parser is configured.  *I think ideally
>>> >> > this
>>> >> > should be fixed in the Scala sources. *
>>> >> >
>>> >> > The issue with the message formatted 300 times when all users are
>>> logged
>>> >> > on
>>> >> > still seems to be there, at least the GC log suggests that, but
I
>>> guess
>>> >> > David will attack that soon. I will send around the profiling reports
>>> >> > later
>>> >> > on.
>>> >> >
>>> >> > Regards,
>>> >> > Markus
>>> >> >
>>> >> >
>>> >> > "The best way to predict the future is to invent it" -- Alan Kay
>>> >> >
>>> >
>>> >
>>>
>>
>

Mime
View raw message