commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mark R. Diggory" <mdigg...@latte.harvard.edu>
Subject Re: [math] Very legitimate reasons to draw up Presicison Unit Tests.
Date Mon, 26 May 2003 17:50:30 GMT
The certified values at

http://www.itl.nist.gov/div898/strd/univ/homepage.html

are limited to mean, std and autocoorelation coef.

I will explore tring to use these in unit tests for Univariate and 
StoreUnivariate using your current strategy from BivariateRegressionTest.

I'm going to try to find certified values for higher order moments too. 
Any pointers to known resources would be helpful.

-Mark


Phil Steitz wrote:
> Mark R. Diggory wrote:
> 
>> I'd say that the last few conversations have really pointed to the 
>> need to have precision Unit testing available as a means of testing 
>> the accuracy of different approaches. Is there a "standardized 
>> consistent way" we can approach Precision Unit Testing?
>>
>> I think we can consider the possibility that we may implement 
>> interfaces like Univariate using different strategies with their own 
>> merits (Efficiency, Memory Usage, Accuracy). Ideally it would be great 
>> to have strong results showing conditions where these algorithms fail 
>> to be accurate (or also fail to be useful from a memory consumption).
>>
>> These tests can be expanded out to include TestStatistics, T-Tests, 
>> Matrix computations.
>>
>> So I see this as an important hurdle to validating the usage of our 
>> library.
>>
>> I know this was brought up earlier by Robert, Phil of Tim. I can't 
>> find the email. But I think it should be added to the list of high 
>> priorities. There was a link to some datsets with precalculated 
>> statistics for comparison that could be tested against. Possibly you 
>> could repost it and I'll include it into the list of relevant links to 
>> other resources (along with many of the links that Brent posted as 
>> resources in one of his last emails.
> 
> 
> Look at BivariateRegressionTest.  This test validates the computations 
> against a NIST reference data set available here:
> http://www.itl.nist.gov/div898/strd/.
> 
> All unit tests should include validation against known examples.  There 
> are some gaps in what has been committed and we should fill these. It 
> should be noted, however, that such tests prove nothing -- they just 
> confirm accuracy for the example being tested.  IMHO, the best approach 
> is to combine rigorous unit testing with validation of algorithms and 
> careful code review. I would also see some targeted performance testing 
> as a good idea.
> 
> Phil
> 
>>
>> -Mark
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
>> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>>
> 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org


Mime
View raw message