incubator-esme-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Kohler <markus.koh...@gmail.com>
Subject Re: Memory allocation analysis Episode II
Date Sat, 28 Nov 2009 08:00:56 GMT
Hi Richard,
taking this to the mailing list.
My plan was always to promote ESME to developers by providing them some
valuable information about how to optimize a real world Scala/Lift
application. If we promote the blog post a bit through dzone, reddit etc. we
should be able to get a few thousand page views on a single day. To be
valuable I would have to do a little bit more work, e.g. numbers for the
response time before and after would certainly something people would ask
for. It would also good to clarify the Scala loadXML issue, something I
wanted to do anyway soon.
 A little introduction into memory allocation tracing would also make
sense.
 I could work on this on Monday night, if nothing else is more important.
Any comments here?

Regards,
Markus

"The best way to predict the future is to invent it" -- Alan Kay


On Sat, Nov 28, 2009 at 7:10 AM, Richard Hirsch <hirsch.dick@gmail.com>wrote:

> The bar chart is a great idea. Would be perfect for a blog. We can
> show we had a memory prob with a reference to a particular wiki page.
> Then describe the code change and then put a link to another wiki
> page. If you create the chart and do a very quick write-up, just
> blurbs. I'll blend everything and write the blog.
>
> Thanks.
>
> D.
>
> On Sat, Nov 28, 2009 at 12:30 AM, Markus Kohler <markus.kohler@gmail.com>
> wrote:
> > Hi,
> > -ESME out of svn 27.11.2009 running on jetty default setting. Turned out
> to
> > be only 64 Mbyte
> > - logged all 300+x users
> > - GC graph 1. attachment yellow line indicates 323 users logged on only
> need
> > around 52Mbyte
> > -Still GC issues when sending one message, but not as bad as before
> > Restarted jetty and did a memory allocation tracing profiling session for
> > one message send; details see 2. attachment
> > -Overall 18.6 Mbyte allocated
> > - 9,9 Mbyte in scala.xml.XML$.loadString(java.lang.String) >50% overall,
> see
> > 3. attachment.
> >
> > Maybe we should just start to provide a simple bar chart.
> > eg. we went down from 90-100 to 18,6.
> > I would like to do a cpu profiling session for one message send, but ATM
> > have problems with my network. Intend to post the result to the scala dev
> > list. Maybe there's a workaround and we can use some other Class/method.
> >
> > Greetings,
> > Markus
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > "The best way to predict the future is to invent it" -- Alan Kay
> >
> >
> > On Fri, Nov 27, 2009 at 10:29 PM, Richard Hirsch <hirsch.dick@gmail.com>
> > wrote:
> >>
> >> @David - I don't know whether the Scala community is following these
> >> threads. Maybe, you can point them to Markus' work and ask them to
> >> take a look at the memory usage of "scala.xml.XML$.loadString".
> >>
> >> @Markus if you'll sen me the reports I'll post another wiki entry and
> >> ten blog about it.
> >>
> >>
> >> Thanks.
> >>
> >> D.
> >>
> >>
> >> On Fri, Nov 27, 2009 at 5:28 PM, Markus Kohler <markus.kohler@gmail.com
> >
> >> wrote:
> >> > Hi all,
> >> > In short, We are down from 90Mbyte to *19Mbyte* for sending one
> message
> >> > and
> >> > I believe it should be possible to get under 10Mbyte  without too much
> >> > effort. With 10  Mbyte we would comply to the performance standard of
> >> > some
> >> > major ERP vendor, but honestly I believe we should be able to do even
> >> > better
> >> > ;-)
> >> >
> >> > Out of the 19Mbyte(maybe even only 11, it's bit hard to get
> reproducible
> >> > results) 9,9 are consumed by
> scala.xml.XML$.loadString(java.lang.String)
> >> > mainly because every time the routine is called, a new Parser is
> >> > instantiated. In addition
> >> > javax.xml.parsers.SAXParserFactory.newInstance()
> >> > is called, which hits  (unless that has changed in recent JDK's) the
> >> > file
> >> > system to find out which XML parser is configured.  *I think ideally
> >> > this
> >> > should be fixed in the Scala sources. *
> >> >
> >> > The issue with the message formatted 300 times when all users are
> logged
> >> > on
> >> > still seems to be there, at least the GC log suggests that, but I
> guess
> >> > David will attack that soon. I will send around the profiling reports
> >> > later
> >> > on.
> >> >
> >> > Regards,
> >> > Markus
> >> >
> >> >
> >> > "The best way to predict the future is to invent it" -- Alan Kay
> >> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message