harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Egor Pasko <egor.pa...@gmail.com>
Subject Re: [performance][benchmark] A tool for writing performance test
Date Tue, 22 May 2007 12:45:53 GMT
On the 0x2DE day of Apache Harmony Andrew Zhang wrote:
> On 5/22/07, Sergey Kuksenko <sergey.kuksenko@gmail.com> wrote:
> >
> > Hi Andrew,
> >
> > I have a couple questions about punit.
> >
> > The first - How do punit calculates amount of used memory?
> > Is it simple difference (totalMemory-freeMemory)?
> > Does it take into acoount the amount of used memory before running a
> > bench?
> > Also, for example, on you examples for ArrayList amount of used memory is
> > around of 3M.
> > But for such amount of used memory, obtained values are more related to
> > how
> > GC works, not the real behavior of ArrayList. What do you think?
> 
> 
> Hi Sergey,
> 
> p-unit calls gc before running the test method, and it collects the memory
> consumption every 10ms by default. Yes, it depends on gc, but p-unit follows
> this way because we can not ask our users to call gc in their codes. So the
> memory consumption is the maximum memory consumed during the execution of
> the method.

Andrew, that looks very nice, but induces extra constraints on
usage. For example, I am now in the middle of writing a set of
performance tests to test JITted code performance with regards to loop
versioning optimization. And it appears that disabling GC during JIT
perf tests is a good idea to get smoother and more predictable
results, thus, simplifying performance analysis of the single
component (JIT).

makes sense?

> p-unit is designed to be pluggable, so if you take a look at the code, it's
> all in MemoryWatcher. If you have a good idea how to monitor the memory, you
> can easily write your own and install the watcher to the core. I'm really
> glad to hear some smart ideas about how to monitor memory. :)

Ideally, we could write VM specific methods to estimate eaten
memory. For example, JIT eats a lot for generated machine-level
code. We have to take it into account.

> And the second question  -does p-unit have tools/frameworks/api/etc for
> > creating a multythreaded tests?
> > Not for purposes to test threading, but for purposes to get higher
> > processors utilization on multycore multyprocessor machines.
> 
> 
> ah... good question. I didn't think about it yet. p-unit uses a runner to
> run the test suites. An easy way to do that is use different thread to run
> different test suite. But it may affect the performance data sine you're
> running other tests at meanwhile?
> 
> On 5/20/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > >
> > > Hi all,
> > >
> > > I wrote a little tool for performance test called punit[1]. It's a
> > > framework
> > > for performance test.
> > >
> > > I used punit to run a performance test ArrayList/Vector#add against
> > > Harmony
> > > and RI. The result report is very straightforward.
> > > From the report, you can easliy see that Harmony performs faster in
> > > ArrayList#add, but consumes much more memory.
> > > More details, please refer to
> > >
> > >
> > http://zhanghuangzhu.blogspot.com/2007/05/use-punit-to-write-performance-test.html
> > > .
> > >
> > > I think if we're interested in the performance test, punit may help us
> > in
> > > some degree.
> > > btw, punit supports to run test suite concurrently too. Hopefully, you
> > > like
> > > it!
> > >
> > > [1] punit:  http://p-unit.sourceforge.net/
> > >
> > > --
> > > Best regards,
> > > Andrew Zhang
> > >
> > > http://zhanghuangzhu.blogspot.com/
> > >
> >
> >
> >
> > --
> > Best regards,
> > ---
> > Sergey Kuksenko.
> > Intel Enterprise Solutions Software Division.
> >
> 
> 
> 
> -- 
> Best regards,
> Andrew Zhang
> 
> http://zhanghuangzhu.blogspot.com/

-- 
Egor Pasko


Mime
View raw message