commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sébastien Brisard <sebastien.bris...@m4x.org>
Subject Re: [Math] Simple benchmarking utility
Date Tue, 26 Jul 2011 11:35:53 GMT
Hi,
I'm willing to help on this if you want. Meanwhile, have you had a look 
to existing frameworks, such as japex (http://japex.java.net/)?
Also, there is some interesting stuff on the web
http://www.ibm.com/developerworks/java/library/j-benchmark1/index.html
I have other electronic papers, I'll try to find them.
Best regards,
Sebastien

Le 26/07/11 12:52, Gilles Sadowski a écrit :
> Hello.
>
>>>>> [...]
>>>>>
>>>>> The idea is to have "interleaved" calls to the candidate
>>>>> implementations, so
>>>>> that (hopefully) they will be penalized (or benefit) in the same
>>>>> way
>>>>> by what
>>>>> the JVM is doing (GC or JIT compilation or ...) while the
>>>>> benchmark
>>>>> is
>>>>> running.
>>>>>
>>>>> Does this make sense?
>>>> Could it be merged by the FastMath performance tests Sebb set up ?
>>> I don't think so. If you meant rewriting the
>>> "FastMathTestPerformance" tests
>>> using the proposed utility, I don't think that it is necessary.
>> This was what I meant.
>> If this feature is not used in any existing tests, perhaps it should go in
>> some other directory. Perhaps a new "utilities" or something like that, at the
>> same level as "main" and "test" ?
>>
>> Anyway, if you feel it's useful to have this available around, don't hesitate.
>>
> Well, the first use I had in mind was to provide a agreed on way to base a
> discussion for requests such as "CM's implementation of foo is not
> efficient", and avoid wondering how the reporter got his results. [This
> problem occurs for the MATH-628 issue.]. Then, when the problem reported
> is confirmed, the new implementation will replace the less efficient one in
> CM, so that there won't be any alternative implementation left to compare.
>
> If you agree with the idea of a "standard" benchmark, it would be very much
> necessary that several people have a look at the code: It might be that my
> crude "methodology" is not right, or that there is a bug.
>
> If the code is accepted, then we'll decide where to put it. Even if,
> according to the above, its primary use will not be for long-lived unit
> tests, it might still be useful in order to compare the efficiency of CM's
> algorithms such as the various optimizers. These comparisons could be added
> as performance reports similar to "FastMathTestPerformance".
>
>
> Thanks,
> Gilles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org


Mime
View raw message