harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leo Li" <liyilei1...@gmail.com>
Subject Re: [classlib][general] How to test the performance of Harmony
Date Thu, 25 Jan 2007 08:07:35 GMT
 Sorry, missing one thing.
 The numbers in the list are mseconds to run tests.:)


On 1/25/07, Leo Li <liyilei1979@gmail.com> wrote:
>
> Hi, all:
>      I have just ran the benchmark tests provided by the dacapo. Although
> I just got the result from my machine with default heap size , it
> really suprises me:  Harmony at least has similar performance output as RI.
>
> Environment: windows XP sp2
> Result:
>
>
>
>
> RI
>
> Harmony
>
> *antrl*
>
> 12235
>
> 13000
>
> *bloat*
>
> 21890
>
> Failure: exception thrown out
>
> *chart*
>
> 20672
>
> 17672
>
> *eclipse*
>
> 98829
>
> 74610
>
> *fop*
>
> 4422
>
> 4578
>
> *hsqldb*
>
> Failure: Out of Memory
>
> 9250
>
> *jython*
>
> 17969
>
> Failure: maybe Harmony has different layout from RI
>
> *lusearch*
>
> 17140
>
> 13657
>
> *luindex*
>
> 28687
>
> 24797
>
> *pmd*
>
> 15129
>
> 12312
>
> *xalan*
>
> 28563
>
> Failure: with xalan version conflicts
>
> (The failures on Harmony might be due to configuration.)
>
> Further, I still have some problem:
> 1. Are there performace test suites focused on io,net and nio, where I
> suspect the space to improve exists.
> 2. Is there performance tests that can provide profiling messages to help
> to diagnose hotspot?
>
> Thanks,
> Leo
>
>
>
> On 1/25/07, Leo Li <liyilei1979@gmail.com> wrote:
> >
> >  On 1/25/07, Geir Magnusson Jr. <geir@pobox.com> wrote:
> >
> > >
> > > On Jan 24, 2007, at 11:00 PM, Robin Garner wrote:
> > >
> > > [SNIP]
> > >
> > > > http://cs.anu.edu.au/people/Robin.Garner/dacapo/regression/ - My
> > > > focus is on DaCapo correctness, the performance test isn't
> > > > particularly rigorous.
> > > >
> > > > IMO for harmony tracking SPECjvm98, SPECjbb2005 and dacapo would be
> > > > a good start.
> > > >
> > > > One thing you definitely should do in a dedicated Harmony
> > > > performance test is test across a range of heap sizes.  In a small
> > > > heap GC improvements and allocation efficiency are more important -
> > > > in large heaps locality effects and code quality dominate.
> > > >
> > > > If possible, testing on a range of machines would be good too.
> > >
> > > I'm hoping that by just making this "plug-n-play" in our build-test
> > > CI and reporting system, we'll be running in many places in no time...
> >
> >
> >
> >  +1
> >
> > Add performance testing to auto build system will be a great help for us
> > to monitor the effect of the modification in source code.
> >
> > geir
> > >
> > >
> >
> >
> > --
> > Leo Li
> > China Software Development Lab, IBM
> >
>
>
>
> --
> Leo Li
> China Software Development Lab, IBM
>



-- 
Leo Li
China Software Development Lab, IBM

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message