harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Strigun" <vstri...@gmail.com>
Subject Re: [performance] performance measurement of HDK
Date Wed, 02 Aug 2006 08:14:12 GMT
On 8/2/06, Geir Magnusson Jr <geir@pobox.com> wrote:
>
>
> Vladimir Strigun wrote:
> > Our project is now in a good shape and we started encouraging people
> > to try real applications on top of Harmony. One of the important parts
> > of the feedback will be general impression of stability, reliability
> > and performance of the HDK. We are doing pretty good in fixing bugs
> > and developing new functionality in HDK, and it might be a good time
> > for us to start thinking of performance as well. It might be a little
> > bit preliminary, but do we consider having any performance targets for
> > us? If yes, we may need to focus at least some of our efforts on
> > benchmarking and tuning overall performance of HDK.
>
> Well, I wouldn't say the HDK, as much as Harmony.
>
> >
> > One of the main questions here is what should be the targets for us
> > and how should we measure our performance. There are several ways for
> > measuring performance, such us commercial benchmarks, free benchmarks,
> > application startup, small micro suites, etc. Some of free benchmarks
> > have been mentioned in JIRA issues and dev list, nevertheless at the
> > moment we don't have any goals for performance. In spite of
> > application enabling initiative it might be good to consider publicly
> > available benchmarks as the additional list of the software
> > applications which we would enable on Harmony.
>
> Yep.
>
> >
> > So I suggest to start discussing performance techniques and methods
> > that can be used for comparing performance between RI and HDK. I think
>
> s/HDK/Harmony
>
> > in case we do not consider performance issues, we can get negative
> > feedback from users even if application starts without any errors,
> > exceptions, etc.
> >
> > One of the benchmarks that was mentioned is DaCapo[1]. It's a free
> > open-source benchmark suite and I believe it can be used for regular
> > performance measurement of HDK. I've tried to find other free suites
> > and got the following list:
> >
> > Telco - this one mostly stresses BigInteger/BigDecimal functionality
> > GcOld - the purpose of this one is clear from the name :)
> > SciMark - java benchmark for scientific and numerical computing
> > Linpack java - well-known benchmark solving linear equations The
> > Plasma Benchmark - creates an animated display by continuously summing
> > four sine waves in an applet
> > JavaWorld Benchmark - benchmark for low-level operations: loops,
> > accessing variables, method invocation, arithmetic operators, casting,
> > instantiation, exception handling, thread creation and switching.
> > CaffeineMark 3.0 - low-level benchmark suite, including sieve of
> > Eratosthenes, sorting, logic ops, method invocation, floating point,
> > simple graphics and GUI ops
> > JavaGrande benchmark suite - a set of benchmarks stressing different
> > areas of java.
>
> These are good.  I it would be nice to just hook them into the
> build-test framework as optional parts.

I'll try to hook them into build-test framework. Do you think that all
of them should be added to the framework? I suppose benchmarks should
be downloaded the same way as other dependencies in Harmony?

> >
> > Having in mind that the list of publicly available benchmarks is not
> > too big, sometimes it will be necessary to create micro benches for
> > some of patches (for instance, Harmony-935). IMO micro should be
> > started in case we change some code that the bench covers.
>
> Sure
>
> >
> > Other interesting and possibly more productive way for comparing
> > performance between different implementations are to use non-free
> > benchmarks. For instance, we can use benchmarks from Spec[n], like
> > SpecJVM, SpecJBB, SpecJAppserver. Unfortunately first we should get
> > license for it, but I believe this issue can be solved within the help
> > of companies participating in Harmony :)
>
> Maybe.  What we want to have is those benchmarks available to anyone in
> the project, although we'll take what we can get for now...
>
> geir
>
> >
> > Thoughts? Comments?
> >
> > [1] http://dacapobench.org
> > [2] http://www.spec.org
> >
> > --
> > Vladimir Strigun,
> > Intel Middleware Products Division
> >
> > ---------------------------------------------------------------------
> > Terms of use : http://incubator.apache.org/harmony/mailing.html
> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> >
> >
> >
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message