harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leo Li" <liyilei1...@gmail.com>
Subject Re: [classlib][general] How to test the performance of Harmony
Date Thu, 25 Jan 2007 08:01:24 GMT
Hi, all:
     I have just ran the benchmark tests provided by the dacapo. Although I
just got the result from my machine with default heap size , it
really suprises me:  Harmony at least has similar performance output as RI.

Environment: windows XP sp2
Result:




RI

Harmony

*antrl*

12235

13000

*bloat*

21890

Failure: exception thrown out

*chart*

20672

17672

*eclipse*

98829

74610

*fop*

4422

4578

*hsqldb*

Failure: Out of Memory

9250

*jython*

17969

Failure: maybe Harmony has different layout from RI

*lusearch*

17140

13657

*luindex*

28687

24797

*pmd*

15129

12312

*xalan*

28563

Failure: with xalan version conflicts

(The failures on Harmony might be due to configuration.)

Further, I still have some problem:
1. Are there performace test suites focused on io,net and nio, where I
suspect the space to improve exists.
2. Is there performance tests that can provide profiling messages to help to
diagnose hotspot?

Thanks,
Leo



On 1/25/07, Leo Li <liyilei1979@gmail.com> wrote:
>
>  On 1/25/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
>
> >
> > On Jan 24, 2007, at 11:00 PM, Robin Garner wrote:
> >
> > [SNIP]
> >
> > > http://cs.anu.edu.au/people/Robin.Garner/dacapo/regression/ - My
> > > focus is on DaCapo correctness, the performance test isn't
> > > particularly rigorous.
> > >
> > > IMO for harmony tracking SPECjvm98, SPECjbb2005 and dacapo would be
> > > a good start.
> > >
> > > One thing you definitely should do in a dedicated Harmony
> > > performance test is test across a range of heap sizes.  In a small
> > > heap GC improvements and allocation efficiency are more important -
> > > in large heaps locality effects and code quality dominate.
> > >
> > > If possible, testing on a range of machines would be good too.
> >
> > I'm hoping that by just making this "plug-n-play" in our build-test
> > CI and reporting system, we'll be running in many places in no time...
>
>
>
>  +1
>
> Add performance testing to auto build system will be a great help for us
> to monitor the effect of the modification in source code.
>
> geir
> >
> >
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>



-- 
Leo Li
China Software Development Lab, IBM

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message