commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Steitz" <>
Subject Re: [math] Very legitimate reasons to draw up Presicison Unit Tests.
Date Tue, 27 May 2003 03:10:10 GMT
Mark R. Diggory wrote:
> Phil Steitz wrote:
>> O'brien, Tim wrote:
>>> On Mon, 2003-05-26 at 18:27, Al Chou wrote:
>>>> How about adding test cases for analytically exact scenarios that an 
>>>> algorithm
>>>> should be able to get correct to within machine precision?  I always 
>>>> had to
>>>> start with those in my research.
>>> I think we've found a good reference in the few certified results from
>>> NIST.  I'd be interested in knowing if there are other freely available
>>> datasets.  Al, you had previously submitted a list of resources, could
>>> you do us the favor of submitting those as a patch for developer.xml?
>>> Tim
>> I agree that we need to pull the references list together. I will also 
>> submit some patches for this.
>> While I think that certified computations are good when we can do 
>> them, I think the following are as important or more:
>> * All implementations should be clearly documented, and wherever
>>   possible we should use standard algorithms with well-known numerical
>>   properties. The matrix LU decomposition discussion is an example of
>>   this.
>> * Support multiple algorithms when appropriate and let the users decide.
>>   For example, we will likely want to add an alternative to Cholesky
>>   decomposition and allow the user to choose.
>> * Prefer verified computations to use in unit tests, but use whatever we
>>   can get our hands on.  It is 1000 times more important to verify that
>>   the algorithm has been correctly implemented than to try to verify
>>   empirically that our code + whatever the JDK does + special
>>   characteristics of the data + our choice of algorithm = numbers
>>   accurate to within some multiple of machine epsilon.
>> Don't get me wrong -- I agree that numerical accuracy is very 
>> important, but unfortumately numerical analysis is not an empirical 
>> science. Analysis of algorithms, good functional testing and code 
>> review are what will get us where we want to be, IMHO.
>> Phil
> I agree with everything you've said, but having datasets and expected 
> results does make simply testing that the algorithm has been correctly 
> implemented, yes they don't neccessarily need to be Certified. Just 
> acurate to the current expectations of the algorithm.

Yeah, that's what I meant. My experience has been that most 
implementation blunders result in *large* discrepancies which can be 
caught with simple unit tests based on examples from books, web 
examples, etc. or even examples computed using other software.  I have 
toyed with the idea of running some tests using R to get stats to 
compare against.

> p.s. I am working on a Unit test for Univariate that should allow us to 
> validate that algorithms for StoredUnivariate and Univariate.
This will be very useful, especially as we consider alternatives to the 
simple formulas being used today.

> -Mark
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message