harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Mazzocchi <stef...@apache.org>
Subject Re: [performance] a few early benchmarks
Date Sat, 18 Nov 2006 18:30:28 GMT
Alexei Fedotov wrote:
> Stefano,
> That is great. Thank you for a nice comparison!

My pleasure, really. I'm a sucker for stats, charts and maps ;-)

> Andrew Zhang wrote,
>> How about the performance of Harmony classlib + IBM VME?
> 
> What does IBM/J9 mean in your list? Does it use Harmony class libraries?

No, IBM/J9 uses its own classlib.

> On 11/18/06, Andrew Zhang <zhanghuangzhu@gmail.com> wrote:
>> On 11/17/06, Stefano Mazzocchi <stefano@apache.org> wrote:
>> >
>> > There are lies, damn lies and benchmarks.... which don't really tell
>> you
>> > if an implementation of a program is *faster* but at least it tells you
>> > where you're at.
>> >
>> > So, as Geir managed to get the DSO linking problem go away in DRLVM, I
>> > was able to start running some benchmarks.
>> >
>> > The machine is the following:
>> >
>> > Linux harmony-em64t 2.6.15-27-amd64-generic #1 SMP PREEMPT Sat Sep 16
>> > 01:50:50 UTC 2006 x86_64 GNU/Linux
>> >
>> > dual Intel(R) Pentium(R) D CPU 3.20GHz
>> > bogomips 6410.31 (per CPU)
>> >
>> > There is nothing else running on the machine (load is 0.04 at the time
>> > of testing).
>> >
>> > The various virtual machines tested are:
>> >
>> > harmony
>> > -------
>> > Apache Harmony Launcher : (c) Copyright 1991, 2006 The Apache Software
>> > Foundation or its licensors, as applicable.
>> > java version "1.5.0"
>> > pre-alpha : not complete or compatible
>> > svn = r476006, (Nov 16 2006), Linux/em64t/gcc 4.0.3, debug build
>>
>>
>> How about the performance of Harmony classlib + IBM VME? It may help
>> us to
>> find out how many spaces are there for us to improve classlib code. :)
>>
>> sun5
>> > ---
>> > java version "1.5.0_09"
>> > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_09-b03)
>> > Java HotSpot(TM) 64-Bit Server VM (build 1.5.0_09-b03, mixed mode)
>> >
>> > sun6
>> > ----
>> > java version "1.6.0-rc"
>> > Java(TM) SE Runtime Environment (build 1.6.0-rc-b104)
>> > Java HotSpot(TM) 64-Bit Server VM (build 1.6.0-rc-b104, mixed mode)
>> >
>> > ibm
>> > ---
>> > java version "1.5.0"
>> > Java(TM) 2 Runtime Environment, Standard Edition (build
>> > pxa64dev-20061002a (SR3) )
>> > IBM J9 VM (build 2.3, J2RE 1.5.0 IBM J9 2.3 Linux amd64-64
>> > j9vmxa6423-20061001 (JIT enabled)
>> > J9VM - 20060915_08260_LHdSMr
>> > JIT  - 20060908_1811_r8
>> > GC   - 20060906_AA)
>> > JCL  - 20061002
>> >
>> > bea
>> > ---
>> > java version "1.5.0_06"
>> > Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_06-b05)
>> > BEA JRockit(R) (build
>> > R26.4.0-63-63688-1.5.0_06-20060626-2259-linux-x86_64, )
>> >
>> >
>> >
>> --------------------------------------------------------------------------
>>
>> >
>> > Test #1: java scimark2 (http://math.nist.gov/scimark2/)
>> >
>> > command: java jnt.scimark2.commandline
>> >
>> > NOTE: bigger number is better
>> >
>> > Sun6
>> > Composite Score: 364.5832265230057
>> > FFT (1024): 220.8458713892794
>> > SOR (100x100):   696.1542342357722
>> > Monte Carlo : 149.37978088875656
>> > Sparse matmult (N=1000, nz=5000): 326.37451873283845
>> > LU (100x100): 430.1617273683819
>> >
>> > BEA
>> > Composite Score: 359.13480378697835
>> > FFT (1024): 303.8746880751562
>> > SOR (100x100):   454.25628897202307
>> > Monte Carlo : 93.23913192138497
>> > Sparse matmult (N=1000, nz=5000): 530.44112637391
>> > LU (100x100): 413.8627835924175
>> >
>> > Sun5
>> > Composite Score: 332.84987587548574
>> > FFT (1024): 216.5144595799027
>> > SOR (100x100):   689.429322146947
>> > Monte Carlo : 25.791262124978065
>> > Sparse matmult (N=1000, nz=5000): 317.5193965699373
>> > LU (100x100): 414.99493895566377
>> >
>> > IBM
>> > Composite Score: 259.8249218693683
>> > FFT (1024): 296.8415012789055
>> > SOR (100x100):   428.974881649179
>> > Monte Carlo : 89.15159857584082
>> > Sparse matmult (N=1000, nz=5000): 144.3524241203982
>> > LU (100x100): 339.8042037225181
>> >
>> > Harmony
>> > Composite Score: 113.65082278962575
>> > FFT (1024): 203.76641991778123
>> > SOR (100x100):   224.37761309236748
>> > Monte Carlo : 9.063866256533116
>> > Sparse matmult (N=1000, nz=5000): 65.4051866327227
>> > LU (100x100): 65.6410280487242
>> >
>> > In this test harmony is clearly lagging behind... at about 30%
>> > performance of the best JVM, it's a little crappy. Please note how
>> FFT's
>> > performance is not so bad awhile monte carlo is pretty bad compared to
>> > BEA or IBM.
>> >
>> > Overall, it seems like there is some serious work to do here to
>> catch up.
>> >
>> >
>> --------------------------------------------------------------------------
>>
>> >
>> > Test 2: Dhrystones
>> (http://www.c-creators.co.jp/okayan/DhrystoneApplet/)
>> >
>> > command: java dhry 100000000
>> >
>> > NOTE: bigger is better
>> >
>> > NB: I modified the code to accept the count at input from the command
>> > line!
>> >
>> > sun6:     8552856 dhrystones/sec
>> > sun5:     6605892
>> > bea:      5678914
>> > harmony:   669734
>> > ibm:       501562
>> >
>> > The performance here is horrific but what's surprising is that J9 is
>> > even worse. No idea what's going on but it seems like something is not
>> > working as it should (in both harmony and J9)
>> >
>> >
>> --------------------------------------------------------------------------
>>
>> >
>> > Test 3: Sieve (part of http://www.sax.de/~adlibit/tya18.tgz)
>> >
>> > command: java Sieve 30
>> >
>> > NB: I modified the test to run for a configurable amount of seconds.
>> >
>> > sun6     8545 sieves/sec
>> > sun5     8364
>> > bea      6174
>> > harmony  1836
>> > ibm       225
>> >
>> > IBM J9 clearly has something wrong on x86_64 but harmony is clearly
>> > lagging behind.
>> >
>> > Stay tuned for more tests.
>> >
>> > --
>> > Stefano.
>> >
>> >
>>
>>
>> -- 
>> Best regards,
>> Andrew Zhang
>>
>>
> 
> 


-- 
Stefano.


Mime
View raw message