Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (herse.apache.org: local policy)
Date: Wed, 22 Nov 2006 19:02:56 +1100
From: Robin Garner <robin.garner@anu.edu.au>
Subject: Re: [performance] harmony 'release' results included
In-reply-to: <4563F9AB.1010502@apache.org>
To: dev@harmony.apache.org
Message-id: <45640430.1080607@anu.edu.au>
MIME-version: 1.0
Content-type: text/plain; charset=ISO-8859-1; format=flowed
Content-transfer-encoding: 7BIT
References: <455E0E5F.1030902@apache.org> <4563C2C7.3000606@anu.edu.au>
 <4563F9AB.1010502@apache.org>
User-Agent: Thunderbird 1.5.0.8 (X11/20061115)

Stefano Mazzocchi wrote:
> Robin Garner wrote:
>> Stefano,
>>
>> Which iteration of the benchmark are you timing ?  
> 
> ehm, the first.
> 
>> This can make a huge
>> difference (our postion is argued in our OOPSLA paper here
>> http://www.dacapobench.org/dacapo-oopsla-2006.pdf).
> 
> I can't seem to be able to read this PDF file, acrobat complains and
> macosx preview reads only parts of it. am I the only one having trouble
> with this?

You could try this link.  sourceforge might be playing up :-|

http://cs.anu.edu.au/people/Robin.Garner/dacapo-oopsla-2006.pdf

Section 5 is the most relevant to what I'm saying.

cheers

> 
>> It would be really informative to see these results plotted for 1st, 2nd
>> and 3rd iteration, for a range of heap sizes.
> 
> I'll see what I can do. I'll try to run dacapo as well.
> 
>> cheers
>>
>> Stefano Mazzocchi wrote:
>>> I've built a 'release' version of Harmony and I'm incorporating the new
>>> results in the previous runs.
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> harmony-debug
>>> -------------
>>> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
>>> Foundation or its licensors, as applicable.
>>> java version "1.5.0"
>>> pre-alpha : not complete or compatible
>>> svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build
>>>
>>> harmony-release
>>> ---------------
>>> Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
>>> Foundation or its licensors, as applicable.
>>> java version "1.5.0"
>>> pre-alpha : not complete or compatible
>>> svn = r476234, (Nov 17 2006), Linux/em64t/gcc 4.0.3, release build
>>>
>>> sun5
>>> ---
>>> java version "1.5.0_09"
>>> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
>>> Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode)
>>>
>>> sun6
>>> ----
>>> java version "1.6.0-rc"
>>> Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
>>> Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode)
>>>
>>> ibm
>>> ---
>>> java version "1.5.0"
>>> Java(TM) 2 Runtime Environment, Standard Edition (build
>>> pxa64dev-20061002a (SR3) )
>>> IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64
>>> j9vmxa6423-20061001 (JIT enabled)
>>> J9VM - 20060915_08260_LHdSMr
>>> JIT  - 20060908_1811_r8
>>> GC   - 20060906_AA)
>>> JCL  - 20061002
>>>
>>> bea
>>> ---
>>> java version "1.5.0_06"
>>> Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
>>> BEA JRockit(R) (build
>>> R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, )
>>>
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> Test #1: java scimark2 (http://math.nist.gov/scimark2/)
>>>
>>> command: java jnt.scimark2.commandline
>>>
>>> Sun6
>>> Composite Score: 364.5832265230057
>>> FFT (1024): 220.8458713892794
>>> SOR (100x100):   696.1542342357722
>>> Monte Carlo : 149.37978088875656
>>> Sparse matmult (N=1000, nz=5000): 326.37451873283845
>>> LU (100x100): 430.1617273683819
>>>
>>> BEA
>>> Composite Score: 359.13480378697835
>>> FFT (1024): 303.8746880751562
>>> SOR (100x100):   454.25628897202307
>>> Monte Carlo : 93.23913192138497
>>> Sparse matmult (N=1000, nz=5000): 530.44112637391
>>> LU (100x100): 413.8627835924175
>>>
>>> Sun5
>>> Composite Score: 332.84987587548574
>>> FFT (1024): 216.5144595799027
>>> SOR (100x100):   689.429322146947
>>> Monte Carlo : 25.791262124978065
>>> Sparse matmult (N=1000, nz=5000): 317.5193965699373
>>> LU (100x100): 414.99493895566377
>>>
>>> IBM
>>> Composite Score: 259.8249218693683
>>> FFT (1024): 296.8415012789055
>>> SOR (100x100):   428.974881649179
>>> Monte Carlo : 89.15159857584082
>>> Sparse matmult (N=1000, nz=5000): 144.3524241203982
>>> LU (100x100): 339.8042037225181
>>>
>>> Harmony-release
>>> Composite Score: 124.6867792398682
>>> FFT (1024): 214.17495491343038
>>> SOR (100x100):   243.36384871876157
>>> Monte Carlo : 14.886614559623572
>>> Sparse matmult (N=1000, nz=5000): 71.70240430885481
>>> LU (100x100): 79.30607369867067
>>>
>>> Harmony-debug
>>> Composite Score: 113.65082278962575
>>> FFT (1024): 203.76641991778123
>>> SOR (100x100):   224.37761309236748
>>> Monte Carlo : 9.063866256533116
>>> Sparse matmult (N=1000, nz=5000): 65.4051866327227
>>> LU (100x100): 65.6410280487242
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/)
>>>
>>> command: java dhry 100000000
>>>
>>> sun6:             8552856 dhrystones/sec
>>> sun5:             6605892
>>> bea:              5678914
>>> harmony-release   1132348
>>> harmony-debug      669734
>>> ibm:               501562
>>>
>>> --------------------------------------------------------------------------
>>>
>>>
>>> Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz)
>>>
>>> command: java Sieve 30
>>>
>>> sun6            8545 sieves/sec
>>> sun5            8364
>>> bea             6174
>>> harmony-release 1848
>>> harmony-debug   1836
>>> ibm              225
>>>
>>>
>>
> 
> 


-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/