commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilles Sadowski <gil...@harfang.homelinux.org>
Subject Re: [Math] Simple benchmarking utility
Date Tue, 26 Jul 2011 10:52:52 GMT
Hello.

> > > > [...]
> > > > 
> > > > The idea is to have "interleaved" calls to the candidate
> > > > implementations, so
> > > > that (hopefully) they will be penalized (or benefit) in the same
> > > > way
> > > > by what
> > > > the JVM is doing (GC or JIT compilation or ...) while the
> > > > benchmark
> > > > is
> > > > running.
> > > > 
> > > > Does this make sense?
> > > 
> > > Could it be merged by the FastMath performance tests Sebb set up ?
> > 
> > I don't think so. If you meant rewriting the
> > "FastMathTestPerformance" tests
> > using the proposed utility, I don't think that it is necessary.
> 
> This was what I meant.
> If this feature is not used in any existing tests, perhaps it should go in
> some other directory. Perhaps a new "utilities" or something like that, at the
> same level as "main" and "test" ?
> 
> Anyway, if you feel it's useful to have this available around, don't hesitate.
> 

Well, the first use I had in mind was to provide a agreed on way to base a
discussion for requests such as "CM's implementation of foo is not
efficient", and avoid wondering how the reporter got his results. [This
problem occurs for the MATH-628 issue.]. Then, when the problem reported
is confirmed, the new implementation will replace the less efficient one in
CM, so that there won't be any alternative implementation left to compare.

If you agree with the idea of a "standard" benchmark, it would be very much
necessary that several people have a look at the code: It might be that my
crude "methodology" is not right, or that there is a bug.

If the code is accepted, then we'll decide where to put it. Even if,
according to the above, its primary use will not be for long-lived unit
tests, it might still be useful in order to compare the efficiency of CM's
algorithms such as the various optimizers. These comparisons could be added
as performance reports similar to "FastMathTestPerformance".


Thanks,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message