commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gilles Sadowski <gil...@harfang.homelinux.org>
Subject Re: [Math] Simple benchmarking utility
Date Fri, 29 Jul 2011 10:23:00 GMT
Hello.

> >
> >>>>> [...]
> >>>>>
> >>>>> The idea is to have "interleaved" calls to the candidate
> >>>>> implementations, so
> >>>>> that (hopefully) they will be penalized (or benefit) in the same
> >>>>> way
> >>>>> by what
> >>>>> the JVM is doing (GC or JIT compilation or ...) while the
> >>>>> benchmark
> >>>>> is
> >>>>> running.
> >>>>>
> >>>>> Does this make sense?
> >>>> Could it be merged by the FastMath performance tests Sebb set up ?
> >>> I don't think so. If you meant rewriting the
> >>> "FastMathTestPerformance" tests
> >>> using the proposed utility, I don't think that it is necessary.
> >> This was what I meant.
> >> If this feature is not used in any existing tests, perhaps it should go in
> >> some other directory. Perhaps a new "utilities" or something like that, at the
> >> same level as "main" and "test" ?
> >>
> >> Anyway, if you feel it's useful to have this available around, don't hesitate.
> >>
> > Well, the first use I had in mind was to provide a agreed on way to base a
> > discussion for requests such as "CM's implementation of foo is not
> > efficient", and avoid wondering how the reporter got his results. [This
> > problem occurs for the MATH-628 issue.]. Then, when the problem reported
> > is confirmed, the new implementation will replace the less efficient one in
> > CM, so that there won't be any alternative implementation left to compare.
> >
> > If you agree with the idea of a "standard" benchmark, it would be very much
> > necessary that several people have a look at the code: It might be that my
> > crude "methodology" is not right, or that there is a bug.
> >
> > If the code is accepted, then we'll decide where to put it. Even if,
> > according to the above, its primary use will not be for long-lived unit
> > tests, it might still be useful in order to compare the efficiency of CM's
> > algorithms such as the various optimizers. These comparisons could be added
> > as performance reports similar to "FastMathTestPerformance".
> 
> +1 to include it.  I would say start by putting it in a top level
> package of its own - say, "benchmark" in src/test/java.  That way we
> can use it in test classes or experimentation that we do using test
> classes to set up benchmarks.

Isn't it "safer" to put it in package "o.a.c.m" (under "src/test/java")?
I was thinking of "PerfTestUtils" for the class name.

> If it evolves into a generically
> useful microbenchmark generator, we can talk about moving it
> src/main.  Thanks for doing this.

I didn't think that this utility would ever move to "main", as it's just for
internal testing.


Regards,
Gilles

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message