harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geir Magnusson Jr <g...@pobox.com>
Subject Re: [performance] performance measurement of HDK
Date Wed, 02 Aug 2006 05:09:01 GMT


Vladimir Strigun wrote:
> Our project is now in a good shape and we started encouraging people
> to try real applications on top of Harmony. One of the important parts
> of the feedback will be general impression of stability, reliability
> and performance of the HDK. We are doing pretty good in fixing bugs
> and developing new functionality in HDK, and it might be a good time
> for us to start thinking of performance as well. It might be a little
> bit preliminary, but do we consider having any performance targets for
> us? If yes, we may need to focus at least some of our efforts on
> benchmarking and tuning overall performance of HDK.

Well, I wouldn't say the HDK, as much as Harmony.

> 
> One of the main questions here is what should be the targets for us
> and how should we measure our performance. There are several ways for
> measuring performance, such us commercial benchmarks, free benchmarks,
> application startup, small micro suites, etc. Some of free benchmarks
> have been mentioned in JIRA issues and dev list, nevertheless at the
> moment we don't have any goals for performance. In spite of
> application enabling initiative it might be good to consider publicly
> available benchmarks as the additional list of the software
> applications which we would enable on Harmony.

Yep.

> 
> So I suggest to start discussing performance techniques and methods
> that can be used for comparing performance between RI and HDK. I think

s/HDK/Harmony

> in case we do not consider performance issues, we can get negative
> feedback from users even if application starts without any errors,
> exceptions, etc.
> 
> One of the benchmarks that was mentioned is DaCapo[1]. It's a free
> open-source benchmark suite and I believe it can be used for regular
> performance measurement of HDK. I've tried to find other free suites
> and got the following list:
> 
> Telco - this one mostly stresses BigInteger/BigDecimal functionality
> GcOld - the purpose of this one is clear from the name :)
> SciMark - java benchmark for scientific and numerical computing
> Linpack java - well-known benchmark solving linear equations The
> Plasma Benchmark - creates an animated display by continuously summing
> four sine waves in an applet
> JavaWorld Benchmark - benchmark for low-level operations: loops,
> accessing variables, method invocation, arithmetic operators, casting,
> instantiation, exception handling, thread creation and switching.
> CaffeineMark 3.0 - low-level benchmark suite, including sieve of
> Eratosthenes, sorting, logic ops, method invocation, floating point,
> simple graphics and GUI ops
> JavaGrande benchmark suite - a set of benchmarks stressing different
> areas of java.

These are good.  I it would be nice to just hook them into the
build-test framework as optional parts.

> 
> Having in mind that the list of publicly available benchmarks is not
> too big, sometimes it will be necessary to create micro benches for
> some of patches (for instance, Harmony-935). IMO micro should be
> started in case we change some code that the bench covers.

Sure

> 
> Other interesting and possibly more productive way for comparing
> performance between different implementations are to use non-free
> benchmarks. For instance, we can use benchmarks from Spec[n], like
> SpecJVM, SpecJBB, SpecJAppserver. Unfortunately first we should get
> license for it, but I believe this issue can be solved within the help
> of companies participating in Harmony :)

Maybe.  What we want to have is those benchmarks available to anyone in
the project, although we'll take what we can get for now...

geir

> 
> Thoughts? Comments?
> 
> [1] http://dacapobench.org
> [2] http://www.spec.org
> 
> -- 
> Vladimir Strigun,
> Intel Middleware Products Division
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> 
> 
> 

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message