harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Zhang" <zhanghuang...@gmail.com>
Subject Re: [performance][benchmark] A tool for writing performance test
Date Tue, 22 May 2007 12:56:47 GMT
On 22 May 2007 05:45:53 -0700, Egor Pasko <egor.pasko@gmail.com> wrote:
>
> On the 0x2DE day of Apache Harmony Andrew Zhang wrote:
> > On 5/22/07, Sergey Kuksenko <sergey.kuksenko@gmail.com> wrote:
> > >
> > > Hi Andrew,
> > >
> > > I have a couple questions about punit.
> > >
> > > The first - How do punit calculates amount of used memory?
> > > Is it simple difference (totalMemory-freeMemory)?
> > > Does it take into acoount the amount of used memory before running a
> > > bench?
> > > Also, for example, on you examples for ArrayList amount of used memory
> is
> > > around of 3M.
> > > But for such amount of used memory, obtained values are more related
> to
> > > how
> > > GC works, not the real behavior of ArrayList. What do you think?
> >
> >
> > Hi Sergey,
> >
> > p-unit calls gc before running the test method, and it collects the
> memory
> > consumption every 10ms by default. Yes, it depends on gc, but p-unit
> follows
> > this way because we can not ask our users to call gc in their codes. So
> the
> > memory consumption is the maximum memory consumed during the execution
> of
> > the method.
>
> Andrew, that looks very nice, but induces extra constraints on
> usage. For example, I am now in the middle of writing a set of
> performance tests to test JITted code performance with regards to loop
> versioning optimization. And it appears that disabling GC during JIT
> perf tests is a good idea to get smoother and more predictable
> results, thus, simplifying performance analysis of the single
> component (JIT).
>
> makes sense?



 Hi Sergey,

ah... sorry, I miss something in my previous mail. It returns the relative
value (maximun - the start point).

Is there still any problem with your scenario?

> p-unit is designed to be pluggable, so if you take a look at the code,
> it's
> > all in MemoryWatcher. If you have a good idea how to monitor the memory,
> you
> > can easily write your own and install the watcher to the core. I'm
> really
> > glad to hear some smart ideas about how to monitor memory. :)
>
> Ideally, we could write VM specific methods to estimate eaten
> memory. For example, JIT eats a lot for generated machine-level
> code. We have to take it into account.
>
> > And the second question  -does p-unit have tools/frameworks/api/etc for
> > > creating a multythreaded tests?
> > > Not for purposes to test threading, but for purposes to get higher
> > > processors utilization on multycore multyprocessor machines.
> >
> >
> > ah... good question. I didn't think about it yet. p-unit uses a runner
> to
> > run the test suites. An easy way to do that is use different thread to
> run
> > different test suite. But it may affect the performance data sine you're
> > running other tests at meanwhile?
> >
> > On 5/20/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > > >
> > > > Hi all,
> > > >
> > > > I wrote a little tool for performance test called punit[1]. It's a
> > > > framework
> > > > for performance test.
> > > >
> > > > I used punit to run a performance test ArrayList/Vector#add against
> > > > Harmony
> > > > and RI. The result report is very straightforward.
> > > > From the report, you can easliy see that Harmony performs faster in
> > > > ArrayList#add, but consumes much more memory.
> > > > More details, please refer to
> > > >
> > > >
> > >
> http://zhanghuangzhu.blogspot.com/2007/05/use-punit-to-write-performance-test.html
> > > > .
> > > >
> > > > I think if we're interested in the performance test, punit may help
> us
> > > in
> > > > some degree.
> > > > btw, punit supports to run test suite concurrently too. Hopefully,
> you
> > > > like
> > > > it!
> > > >
> > > > [1] punit:  http://p-unit.sourceforge.net/
> > > >
> > > > --
> > > > Best regards,
> > > > Andrew Zhang
> > > >
> > > > http://zhanghuangzhu.blogspot.com/
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > ---
> > > Sergey Kuksenko.
> > > Intel Enterprise Solutions Software Division.
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew Zhang
> >
> > http://zhanghuangzhu.blogspot.com/
>
> --
> Egor Pasko
>
>


-- 
Best regards,
Andrew Zhang

http://zhanghuangzhu.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message