harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Zhang" <zhanghuang...@gmail.com>
Subject Re: [buildtest][harness] performance measurement infrastructure
Date Fri, 22 Jun 2007 11:44:05 GMT
On 6/22/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
>
> Andrew,
> This is very interesting. Thank you for the reference.


Hi Alexei,  please see my comments inline

I've got few questions trying to understand your framework.
> 1. Why an interface Watcher extends Serializable?


Because it's for performance comparison between different vms. The runner
will be serialized, so that the vm can load the right runner. You can
consider it like serializing the configuration for the vm.

2. Is there any project which uses p-unit for testing? Why have you started
> it?


Polepos (https://sourceforge.net/projects/polepos) is going to refactor its
code to use p-unit.

And I know serveral community members use p-unit to test concurrency
performance(by running the existing JUnit tests).

I started it because I found that I can't do something as I want with JUnit,
:) like concurrency, performance test. Why do I have to start thread, and
join, and check the result at the end of join everytime? why do I need to
use System.currentMilliseconds and sysout... :)  So I wrote p-unit.

3. Running tests in several VM is great, but how could I compare these
> results?


There're serveral build-in reports in p-unit. The most straightforward one
is pdf format, which contains performance chart. The results of different vm
for the same method are shown in the same page, so that you can know which
one is better easily. It's also easy to plugin your own report by
implementing  PUnitEventListener.

4. If I need to increase a parameter until the test fails and then
> report the final value as a test performance metrics how this should
> be done in p-unit?


hm... this functionality is not supported in p-unit....

Thanks!

On 6/22/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > On 6/21/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > >
> > > Hello Vladimir,
> > >
> > > Could yout please give few hints how one should write performance
> > > tests using the harness? I found an option "-performance" but cannot
> > > see anything else. I wonder how a testcase could report a performance
> > > metrics.
> > >
> > > Actually I'm thinking now of reviving comparative stress testing
> approach.
> >
> >
> > Hi Alexei,
> >
> > Do you have a chance to look at p-unit, (http://p-unit.sourceforge.net/
> )?
> >
> > I think it fulfills most of the requests listed in the documents.
> >
> > Another advantage of p-unit is that it can compare the performance
> between
> > different vms.
> >
> > One problem may be the license (GPL now, going to change the license to
> GPL
> > v3). If we really take it serious, I can relicense it immediately.
> >
> > [1]
> > >
> http://mail-archives.apache.org/mod_mbox/harmony-dev/200605.mbox/%3C6694B22B6436BC43B429958787E4549802012354@mssmsx402nb%3E
> > >
> > > --
> > > With best regards,
> > > Alexei,
> > > ESSD, Intel
> > >
> >
> >
> >
> > --
> > Best regards,
> > Andrew Zhang
> >
> > http://zhanghuangzhu.blogspot.com/
> >
>
>
> --
> With best regards,
> Alexei,
> ESSD, Intel
>



-- 
Best regards,
Andrew Zhang

http://zhanghuangzhu.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message