harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robin Garner <robin.gar...@anu.edu.au>
Subject Re: [classlib][general] How to test the performance of Harmony
Date Thu, 25 Jan 2007 04:00:40 GMT
Geir Magnusson Jr. wrote:
> 
> On Jan 24, 2007, at 10:34 PM, Leo Li wrote:
> 
>> Hi, all:
>>     With the progress of Harmony project, I think, performance is 
>> gradually
>> becoming an important feature for our real customers. Besides the 
>> stability
>> and compatibility, users would incline to choose the one with higher
>> speed. Aside from VM, classlib itself also plays a significant role in
>> performance.
>>    Although we now know some issues that are performance related, in a 
>> long
>> run, we need performace test results to pick out the hotspot in Harmony
>> project. Without data, we are like blind men running in the dark, in the
>> area of performance.
>>    Is there any advice about the tools to test the performance of 
>> Harmony?
>>    Actually, if my remember is right, some guys have actually run the
>> performance tests. (But I forget where the mail is.)Maybe we can post the
>> result on Harmony wiki thus we can have a baseline to improve our code.:)
> 
> Good stuff.
> 
> I think that Robin has been tracking using DaCapo.  One of my goals 
> near-term for build-test is to add Da Capo to the CC run, so we can 
> start recording that information daily, looking for regressions, and 
> seeing how we improve.
> 
> We should also do the same for other free and non-free tests (like SPEC).
> 
> The real question for me is how we choose one and start profiling to 
> find the hotspots...
> 
> geir

http://cs.anu.edu.au/people/Robin.Garner/dacapo/regression/ - My focus 
is on DaCapo correctness, the performance test isn't particularly 
rigorous.

IMO for harmony tracking SPECjvm98, SPECjbb2005 and dacapo would be a 
good start.

One thing you definitely should do in a dedicated Harmony performance 
test is test across a range of heap sizes.  In a small heap GC 
improvements and allocation efficiency are more important - in large 
heaps locality effects and code quality dominate.

If possible, testing on a range of machines would be good too.

-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Mime
View raw message