harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Mazzocchi <stef...@apache.org>
Subject [performance] a few early benchmarks
Date Fri, 17 Nov 2006 05:26:17 GMT
There are lies, damn lies and benchmarks.... which don't really tell you
if an implementation of a program is *faster* but at least it tells you
where you're at.

So, as Geir managed to get the DSO linking problem go away in DRLVM, I
was able to start running some benchmarks.

The machine is the following:

Linux harmony-em64t 2.6.15-27-amd64-generic #1 SMP PREEMPT Sat Sep 16
01:50:50 UTC 2006 x86_64 GNU/Linux

dual Intel(R) Pentium(R) D CPU 3.20GHz
bogomips 6410.31 (per CPU)

There is nothing else running on the machine (load is 0.04 at the time
of testing).

The various virtual machines tested are:

harmony
-------
Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
Foundation or its licensors, as applicable.
java version "1.5.0"
pre-alpha : not complete or compatible
svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build

sun5
---
java version "1.5.0_09"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode)

sun6
----
java version "1.6.0-rc"
Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode)

ibm
---
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build
pxa64dev-20061002a (SR3) )
IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64
j9vmxa6423-20061001 (JIT enabled)
J9VM - 20060915_08260_LHdSMr
JIT  - 20060908_1811_r8
GC   - 20060906_AA)
JCL  - 20061002

bea
---
java version "1.5.0_06"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
BEA JRockit(R) (build
R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, )


--------------------------------------------------------------------------

Test #1: java scimark2 (http://math.nist.gov/scimark2/)

command: java jnt.scimark2.commandline

NOTE: bigger number is better

Sun6
Composite Score: 364.5832265230057
FFT (1024): 220.8458713892794
SOR (100x100):   696.1542342357722
Monte Carlo : 149.37978088875656
Sparse matmult (N=1000, nz=5000): 326.37451873283845
LU (100x100): 430.1617273683819

BEA
Composite Score: 359.13480378697835
FFT (1024): 303.8746880751562
SOR (100x100):   454.25628897202307
Monte Carlo : 93.23913192138497
Sparse matmult (N=1000, nz=5000): 530.44112637391
LU (100x100): 413.8627835924175

Sun5
Composite Score: 332.84987587548574
FFT (1024): 216.5144595799027
SOR (100x100):   689.429322146947
Monte Carlo : 25.791262124978065
Sparse matmult (N=1000, nz=5000): 317.5193965699373
LU (100x100): 414.99493895566377

IBM
Composite Score: 259.8249218693683
FFT (1024): 296.8415012789055
SOR (100x100):   428.974881649179
Monte Carlo : 89.15159857584082
Sparse matmult (N=1000, nz=5000): 144.3524241203982
LU (100x100): 339.8042037225181

Harmony
Composite Score: 113.65082278962575
FFT (1024): 203.76641991778123
SOR (100x100):   224.37761309236748
Monte Carlo : 9.063866256533116
Sparse matmult (N=1000, nz=5000): 65.4051866327227
LU (100x100): 65.6410280487242

In this test harmony is clearly lagging behind... at about 30%
performance of the best JVM, it's a little crappy. Please note how FFT's
performance is not so bad awhile monte carlo is pretty bad compared to
BEA or IBM.

Overall, it seems like there is some serious work to do here to catch up.

--------------------------------------------------------------------------

Test 2: Dhrystones (http://www.c-creators.co.jp/okayan/DhrystoneApplet/)

command: java dhry 100000000

NOTE: bigger is better

NB: I modified the code to accept the count at input from the command line!

sun6:     8552856 dhrystones/sec
sun5:     6605892
bea:      5678914
harmony:   669734
ibm:       501562

The performance here is horrific but what's surprising is that J9 is
even worse. No idea what's going on but it seems like something is not
working as it should (in both harmony and J9)

--------------------------------------------------------------------------

Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz)

command: java Sieve 30

NB: I modified the test to run for a configurable amount of seconds.

sun6     8545 sieves/sec
sun5     8364
bea      6174
harmony  1836
ibm       225

IBM J9 clearly has something wrong on x86_64 but harmony is clearly
lagging behind.

Stay tuned for more tests.

-- 
Stefano.


Mime
View raw message