harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Strigun" <vstri...@gmail.com>
Subject [performance] performance measurement of HDK
Date Tue, 01 Aug 2006 12:55:53 GMT
Our project is now in a good shape and we started encouraging people
to try real applications on top of Harmony. One of the important parts
of the feedback will be general impression of stability, reliability
and performance of the HDK. We are doing pretty good in fixing bugs
and developing new functionality in HDK, and it might be a good time
for us to start thinking of performance as well. It might be a little
bit preliminary, but do we consider having any performance targets for
us? If yes, we may need to focus at least some of our efforts on
benchmarking and tuning overall performance of HDK.

One of the main questions here is what should be the targets for us
and how should we measure our performance. There are several ways for
measuring performance, such us commercial benchmarks, free benchmarks,
application startup, small micro suites, etc. Some of free benchmarks
have been mentioned in JIRA issues and dev list, nevertheless at the
moment we don't have any goals for performance. In spite of
application enabling initiative it might be good to consider publicly
available benchmarks as the additional list of the software
applications which we would enable on Harmony.

So I suggest to start discussing performance techniques and methods
that can be used for comparing performance between RI and HDK. I think
in case we do not consider performance issues, we can get negative
feedback from users even if application starts without any errors,
exceptions, etc.

One of the benchmarks that was mentioned is DaCapo[1]. It's a free
open-source benchmark suite and I believe it can be used for regular
performance measurement of HDK. I've tried to find other free suites
and got the following list:

Telco - this one mostly stresses BigInteger/BigDecimal functionality
GcOld - the purpose of this one is clear from the name :)
SciMark - java benchmark for scientific and numerical computing
Linpack java - well-known benchmark solving linear equations The
Plasma Benchmark - creates an animated display by continuously summing
four sine waves in an applet
JavaWorld Benchmark - benchmark for low-level operations: loops,
accessing variables, method invocation, arithmetic operators, casting,
instantiation, exception handling, thread creation and switching.
CaffeineMark 3.0 - low-level benchmark suite, including sieve of
Eratosthenes, sorting, logic ops, method invocation, floating point,
simple graphics and GUI ops
JavaGrande benchmark suite - a set of benchmarks stressing different
areas of java.

Having in mind that the list of publicly available benchmarks is not
too big, sometimes it will be necessary to create micro benches for
some of patches (for instance, Harmony-935). IMO micro should be
started in case we change some code that the bench covers.

Other interesting and possibly more productive way for comparing
performance between different implementations are to use non-free
benchmarks. For instance, we can use benchmarks from Spec[n], like
SpecJVM, SpecJBB, SpecJAppserver. Unfortunately first we should get
license for it, but I believe this issue can be solved within the help
of companies participating in Harmony :)

Thoughts? Comments?

[1] http://dacapobench.org
[2] http://www.spec.org

Vladimir Strigun,
Intel Middleware Products Division

Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org

View raw message