harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Zhang" <zhanghuang...@gmail.com>
Subject Re: [performance][benchmark] A tool for writing performance test
Date Tue, 22 May 2007 12:58:48 GMT
On 5/22/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
>
>
>
> On 22 May 2007 05:45:53 -0700, Egor Pasko <egor.pasko@gmail.com> wrote:
> >
> > On the 0x2DE day of Apache Harmony Andrew Zhang wrote:
> > > On 5/22/07, Sergey Kuksenko <sergey.kuksenko@gmail.com> wrote:
> > > >
> > > > Hi Andrew,
> > > >
> > > > I have a couple questions about punit.
> > > >
> > > > The first - How do punit calculates amount of used memory?
> > > > Is it simple difference (totalMemory-freeMemory)?
> > > > Does it take into acoount the amount of used memory before running a
> >
> > > > bench?
> > > > Also, for example, on you examples for ArrayList amount of used
> > memory is
> > > > around of 3M.
> > > > But for such amount of used memory, obtained values are more related
> > to
> > > > how
> > > > GC works, not the real behavior of ArrayList. What do you think?
> > >
> > >
> > > Hi Sergey,
> > >
> > > p-unit calls gc before running the test method, and it collects the
> > memory
> > > consumption every 10ms by default. Yes, it depends on gc, but p-unit
> > follows
> > > this way because we can not ask our users to call gc in their codes.
> > So the
> > > memory consumption is the maximum memory consumed during the execution
> > of
> > > the method.
> >
> > Andrew, that looks very nice, but induces extra constraints on
> > usage. For example, I am now in the middle of writing a set of
> > performance tests to test JITted code performance with regards to loop
> > versioning optimization. And it appears that disabling GC during JIT
> > perf tests is a good idea to get smoother and more predictable
> > results, thus, simplifying performance analysis of the single
> > component (JIT).
> >
> > makes sense?
>
>
>
>  Hi Sergey,
>

sorry again, Hi Egor. :)

ah... sorry, I miss something in my previous mail. It returns the relative
> value (maximun - the start point).
>
> Is there still any problem with your scenario?
>
> > p-unit is designed to be pluggable, so if you take a look at the code,
> > it's
> > > all in MemoryWatcher. If you have a good idea how to monitor the
> > memory, you
> > > can easily write your own and install the watcher to the core. I'm
> > really
> > > glad to hear some smart ideas about how to monitor memory. :)
> >
> > Ideally, we could write VM specific methods to estimate eaten
> > memory. For example, JIT eats a lot for generated machine-level
> > code. We have to take it into account.
> >
> > > And the second question  -does p-unit have tools/frameworks/api/etc
> > for
> > > > creating a multythreaded tests?
> > > > Not for purposes to test threading, but for purposes to get higher
> > > > processors utilization on multycore multyprocessor machines.
> > >
> > >
> > > ah... good question. I didn't think about it yet. p-unit uses a runner
> > to
> > > run the test suites. An easy way to do that is use different thread to
> > run
> > > different test suite. But it may affect the performance data sine
> > you're
> > > running other tests at meanwhile?
> > >
> > > On 5/20/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > > > >
> > > > > Hi all,
> > > > >
> > > > > I wrote a little tool for performance test called punit[1]. It's
a
> >
> > > > > framework
> > > > > for performance test.
> > > > >
> > > > > I used punit to run a performance test ArrayList/Vector#add
> > against
> > > > > Harmony
> > > > > and RI. The result report is very straightforward.
> > > > > From the report, you can easliy see that Harmony performs faster
> > in
> > > > > ArrayList#add, but consumes much more memory.
> > > > > More details, please refer to
> > > > >
> > > > >
> > > >
> > http://zhanghuangzhu.blogspot.com/2007/05/use-punit-to-write-performance-test.html
> > > > > .
> > > > >
> > > > > I think if we're interested in the performance test, punit may
> > help us
> > > > in
> > > > > some degree.
> > > > > btw, punit supports to run test suite concurrently too. Hopefully,
> > you
> > > > > like
> > > > > it!
> > > > >
> > > > > [1] punit:  http://p-unit.sourceforge.net/
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrew Zhang
> > > > >
> > > > > http://zhanghuangzhu.blogspot.com/
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > ---
> > > > Sergey Kuksenko.
> > > > Intel Enterprise Solutions Software Division.
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew Zhang
> > >
> > > http://zhanghuangzhu.blogspot.com/
> >
> > --
> > Egor Pasko
> >
> >
>
>
> --
> Best regards,
> Andrew Zhang
>
> http://zhanghuangzhu.blogspot.com/
>



-- 
Best regards,
Andrew Zhang

http://zhanghuangzhu.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message