openoffice-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Herbert Duerr <...@apache.org>
Subject Re: Example of spreadsheet formula testing
Date Wed, 28 Aug 2013 08:51:43 GMT
On 27.08.2013 15:29, Rob Weir wrote:
> On Tue, Aug 27, 2013 at 8:44 AM, Herbert Duerr <hdu@apache.org> wrote:
>> [...]
>> A good idea. I just wrote a task [1] and a script [2] that takes a slightly
>> modified version of Regina's sample document and checks whether all of its
>> tests pass. This script is now part of the Functional Verification Test
>> (a.k.a. FVT).
>>
>> [1] https://issues.apache.org/ooo/show_bug.cgi?id=123119
>> [2] http://svn.apache.org/r1517802
>>
>> Adding further test spreadsheets is simple. They need a "TestID" row and a
>> "TestOK" row like [3]. Such new test spreadsheets have to be added to [4]'s
>> testFormulaDocs() method to be picked up.
>>
>
> Nice.  If we can have test cases written by non-programmers, and then
> automated testing of these sheets, then we'll have the best of both
> worlds.
>
> I assume we'd want to agree on a test template.

Here is a suggested test template. It is small and fulfills the 
requirements of containing exactly one sheet and having two columns 
named "TestID" and "TestOK". Please use a monospaced font for your mail 
client to view it:

          A           B       C        D           E           F
1      TestID    Addend1 Addend2 ExpectedSum CalculatedSum  TestOK
2   "OnePlusOne"    1       1         2         =B2+C2     =(D2=E2)
3      "Seven"      3       4         7         =B3+C3     =(D3=E3)

>  For example, one
> approach is to bootstrap the tests like this:
>
> 1) Have the fundamental test cases for IF(), TRUE(), FALSE() and a
> handful of other core functions written in Java, not requiring a test
> spreadsheet.

Core functionionality must be checked in one of the pure automatic test 
suites like BVT ("Build Verification Tests") or FVT ("Functional 
Verification Tests") of course.

The idea that additionally to this traditional tests there should also 
be test cases suitable for both manual and automated testing and that 
can be understood, checked and enhanced by non-developers opens options 
for testers and power-users that were not open before to non-developers.

> 2) Then write the test cases for the specialized functions in test
> documents, and reserve a cell (A1 for example) to return 1 if all
> tests in the document pass, 0 otherwise.

The new test script checks the results of each individual sub-test and 
the test log reports each failed test. Having a specialized cell that 
summarizes all the sub-tests into one result is possible of course. For 
the example above a cell =(SUM(F2:F3)=COUNT(F2:F3)) would provide just 
that. It isn't needed for the new automatism though.

The test spreadsheets should mostly be developed to be most useful for 
manual testing and for extending by testers and power users while 
keeping the requirements ("one sheet", "one TestID column" and "one 
TestOK column") for the new automation at a minimum. Adding more 
requirements again would be counterproductive as lowering the barrier of 
entry is the central point of the new mechanism.

> 3) Automate the loading and evaluation of the test documents.
>
> (Since the test documents would depend on correct operation of IF(),
> we cannot use the test documents to test IF() itself)

A build where simple things like IF() don't work should never make it 
into manual testing. Testers should expect things like that to work.

Herbert

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@openoffice.apache.org
For additional commands, e-mail: dev-help@openoffice.apache.org


Mime
View raw message