commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phil Steitz" <p...@steitz.com>
Subject Re: [math] Very legitimate reasons to draw up Presicison Unit Tests.
Date Tue, 27 May 2003 03:10:10 GMT
Mark R. Diggory wrote:
> 
> 
> Phil Steitz wrote:
> 
>> O'brien, Tim wrote:
>>
>>> On Mon, 2003-05-26 at 18:27, Al Chou wrote:
>>>
>>>> How about adding test cases for analytically exact scenarios that an 
>>>> algorithm
>>>> should be able to get correct to within machine precision?  I always 
>>>> had to
>>>> start with those in my research.
>>>
>>>
>>>
>>>
>>> I think we've found a good reference in the few certified results from
>>> NIST.  I'd be interested in knowing if there are other freely available
>>> datasets.  Al, you had previously submitted a list of resources, could
>>> you do us the favor of submitting those as a patch for developer.xml?
>>>
>>> Tim
>>
>>
>>
>> I agree that we need to pull the references list together. I will also 
>> submit some patches for this.
>>
>> While I think that certified computations are good when we can do 
>> them, I think the following are as important or more:
>>
>> * All implementations should be clearly documented, and wherever
>>   possible we should use standard algorithms with well-known numerical
>>   properties. The matrix LU decomposition discussion is an example of
>>   this.
>>
>> * Support multiple algorithms when appropriate and let the users decide.
>>   For example, we will likely want to add an alternative to Cholesky
>>   decomposition and allow the user to choose.
>>
>> * Prefer verified computations to use in unit tests, but use whatever we
>>   can get our hands on.  It is 1000 times more important to verify that
>>   the algorithm has been correctly implemented than to try to verify
>>   empirically that our code + whatever the JDK does + special
>>   characteristics of the data + our choice of algorithm = numbers
>>   accurate to within some multiple of machine epsilon.
>>
>> Don't get me wrong -- I agree that numerical accuracy is very 
>> important, but unfortumately numerical analysis is not an empirical 
>> science. Analysis of algorithms, good functional testing and code 
>> review are what will get us where we want to be, IMHO.
>>
>> Phil
>>
> 
> I agree with everything you've said, but having datasets and expected 
> results does make simply testing that the algorithm has been correctly 
> implemented, yes they don't neccessarily need to be Certified. Just 
> acurate to the current expectations of the algorithm.
> 

Yeah, that's what I meant. My experience has been that most 
implementation blunders result in *large* discrepancies which can be 
caught with simple unit tests based on examples from books, web 
examples, etc. or even examples computed using other software.  I have 
toyed with the idea of running some tests using R to get stats to 
compare against.

> p.s. I am working on a Unit test for Univariate that should allow us to 
> validate that algorithms for StoredUnivariate and Univariate.
> 
This will be very useful, especially as we consider alternatives to the 
simple formulas being used today.

> -Mark
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
> 




---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message