harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject [buildtest][harness][p-unit] performance measurement infrastructure
Date Mon, 25 Jun 2007 19:56:04 GMT
Andrew,

That's great. You are really quick. I was still thinking when your
solution came up. I was able to build the project and run tests one by
one, though I wasn't able to use the perfect Eclipse plugin for JUnit
tests. Probably it could make sense to use just an ordinary junit for
testing p-unit itself.

1. I've just realized that I cannot link stress tests with p-unit
while it is GPLed. So it's better to change a license. GPL with
exception is ok, but it's safer to have BSD-like license. Actually
this is the only comment which prevents me from using punit, all
others are mostly for better understanding of your concept.

2. Do you think "toStop" and "toAbort" from my requirements are the
best terms? We can change them until there is not too much code
developed. A typical use case is the following. I give a test a fair
time to loop (= first timer) and a small type to stop gracefully (=
second timer). The variant named them "toWork", "toAbort". Another
variant which comes to my mind is "toLoop", "toStop". What do you
think?

3. What do you think of using junit.extensions.TestDecorator for
building complex tests on the top of the primitive ones?




On 6/25/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> On 6/22/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> >
> > Andrew,
> > Thanks for your explanations.
> >
> > Let me address the main problem I see to migrate a stress test suite
> > to p-unit. To which extend may I count on your support at further
> > p-unit development? I listed very specific functionality which
> > probably didn't suite well your original idea.
>
>
> Hi Alexei,
>
> Would you please check the latest sample code samples.LoopTestSample in
> punit.sample project? Is it the same as the original idea of toStop,
> toAbort? Thanks!
>
> On 6/22/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > > On 6/22/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > > >
> > > > Andrew,
> > > > This is very interesting. Thank you for the reference.
> > >
> > >
> > > Hi Alexei,  please see my comments inline
> > >
> > > I've got few questions trying to understand your framework.
> > > > 1. Why an interface Watcher extends Serializable?
> > >
> > >
> > > Because it's for performance comparison between different vms. The
> > runner
> > > will be serialized, so that the vm can load the right runner. You can
> > > consider it like serializing the configuration for the vm.
> > >
> > > 2. Is there any project which uses p-unit for testing? Why have you
> > started
> > > > it?
> > >
> > >
> > > Polepos (https://sourceforge.net/projects/polepos) is going to refactor
> > its
> > > code to use p-unit.
> > >
> > > And I know serveral community members use p-unit to test concurrency
> > > performance(by running the existing JUnit tests).
> > >
> > > I started it because I found that I can't do something as I want with
> > JUnit,
> > > :) like concurrency, performance test. Why do I have to start thread,
> > and
> > > join, and check the result at the end of join everytime? why do I need
> > to
> > > use System.currentMilliseconds and sysout... :)  So I wrote p-unit.
> > >
> > > 3. Running tests in several VM is great, but how could I compare these
> > > > results?
> > >
> > >
> > > There're serveral build-in reports in p-unit. The most straightforward
> > one
> > > is pdf format, which contains performance chart. The results of
> > different vm
> > > for the same method are shown in the same page, so that you can know
> > which
> > > one is better easily. It's also easy to plugin your own report by
> > > implementing  PUnitEventListener.
> > >
> > > 4. If I need to increase a parameter until the test fails and then
> > > > report the final value as a test performance metrics how this should
> > > > be done in p-unit?
> > >
> > >
> > > hm... this functionality is not supported in p-unit....
> > >
> > > Thanks!
> > >
> > > On 6/22/07, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
> > > > > On 6/21/07, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > > > > >
> > > > > > Hello Vladimir,
> > > > > >
> > > > > > Could yout please give few hints how one should write performance
> > > > > > tests using the harness? I found an option "-performance" but
> > cannot
> > > > > > see anything else. I wonder how a testcase could report a
> > performance
> > > > > > metrics.
> > > > > >
> > > > > > Actually I'm thinking now of reviving comparative stress testing
> > > > approach.
> > > > >
> > > > >
> > > > > Hi Alexei,
> > > > >
> > > > > Do you have a chance to look at p-unit, (
> > http://p-unit.sourceforge.net/
> > > > )?
> > > > >
> > > > > I think it fulfills most of the requests listed in the documents.
> > > > >
> > > > > Another advantage of p-unit is that it can compare the performance
> > > > between
> > > > > different vms.
> > > > >
> > > > > One problem may be the license (GPL now, going to change the license
> > to
> > > > GPL
> > > > > v3). If we really take it serious, I can relicense it immediately.
> > > > >
> > > > > [1]
> > > > > >
> > > >
> > http://mail-archives.apache.org/mod_mbox/harmony-dev/200605.mbox/%3C6694B22B6436BC43B429958787E4549802012354@mssmsx402nb%3E
> > > > > >
> > > > > > --
> > > > > > With best regards,
> > > > > > Alexei,
> > > > > > ESSD, Intel
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrew Zhang
> > > > >
> > > > > http://zhanghuangzhu.blogspot.com/
> > > > >
> > > >
> > > >
> > > > --
> > > > With best regards,
> > > > Alexei,
> > > > ESSD, Intel
> > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Andrew Zhang
> > >
> > > http://zhanghuangzhu.blogspot.com/
> > >
> >
> >
> > --
> > With best regards,
> > Alexei,
> > ESSD, Intel
> >
>
>
>
> --
> Best regards,
> Andrew Zhang
>
> http://zhanghuangzhu.blogspot.com/
>


-- 
With best regards,
Alexei,
ESSD, Intel

Mime
View raw message