harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mikhail Loenko" <mloe...@gmail.com>
Subject Re: [classlib] millions of rmi tests
Date Wed, 31 May 2006 06:06:05 GMT
Hi Daniel,

I think what we need is tests that verify that our implementation works
according to our design.

As I understand the design for that constructor is as follows: "store all
parameters in private fields and don't do any special for special parameters".

That means that we need just two test methods:
- test that passes 'normal' parameters to the constructor and verifies
private fields
by getXX methods (or with access to internal stuff)
- test that passes 'suspicious' values for all the parameters and verifies that
no exception thrown and private fields are set as expected.

The broad testing you've mentioned does make much sense on design stage:
e.g. if the spec is not clear enough you might want to do a black-box
testing of RI.
For example, for an integer parameter of a method you might want to go
through all the integer values and explore how it works on RI.

But once you explored it and found that e.g. it works this way for all positive
values and that way for all negative values and zero, you do an
appropriate design
and include a small number of unit tests to the test suite (like passing
0, 1, -1, 10, 10, minvalue, maxvalue, couple more). You don't have to
include all the tests that you used for design.

Please let me know what you think

Thanks,
Mikhail


2006/5/30, Daniel Gandara <danielgandara@neosur.com>:
> Mikhail,
>    Let me explain the reasons why we developed this amount of tests;
> We decided to develop implementation independant tests cases which
> will show if a given implementation of the package behaves as it should
> (RI); we also decided to implement them at the same time the package
> was being coded, so no implementation specific detail was known by
> the time the tests were developed.
>   In order to do this we developed:
>  - Unit tests: for them we took different approachs, which were:
>            - broad tests: where we test each public method with a set of
>                                 possible  inputs
>            - specific tests: using 'suspicious' parameters like null
>    unit tests were developed using a mix of automatic code generation
>    and regular coding.
>  - Integration tests: http tunneling and distributed integration tests
>    which provide a more realistic and "smart" test for the package.
>
> All this tests allowed us to check the correct functionality of our
> package.
>
> That said, I will agree with you that at this moment some of this tests
> (mostly basic JUnit tests) might not be very usefull, and I would add
> that more and complex integration tests should be written to check
> the correct functionality of rmi package.
> At this moment we are running all tests (both itc and intel) against
> contributed rmi packages (both itc and intel); I'll send the report as
> soon as I have it.
>
> Thanks,
>
> Daniel
> note: latest tests source code and doc can be found at
>         http://issues.apache.org/jira/browse/HARMONY-473
> btw: there were not millions but a few thousands tests :)
>
> ----- Original Message -----
> From: "Mikhail Loenko" <mloenko@gmail.com>
> To: <harmony-dev@incubator.apache.org>
> Sent: Monday, May 29, 2006 10:01 AM
> Subject: [classlib] millions of rmi tests
>
>
> I've tried to integrate rmi2 tests to rmi module,  and found some odd
> things.
>
> Let's take a look for example at TestActivationGroupDesc.java
>
> it has 5158 test methods, most of which are very similar.
> For example it has 855 tests that invoke constructor with various parameters
> and check that new did not return null and no exception was thrown:
>
> Compare
>
> public void
> testActivationGroupDescStringStringMarshalledObjectPropertiesCommandEnvironment006()
> {
>
>    try{
>        Properties p= new Properties();
>        assertNotNull(msgNotNull, new ActivationGroupDesc(null ,  null ,
>                new MarshalledObject(new Double(23.4))  ,  new Properties()
> ,
>                new ActivationGroupDesc.CommandEnvironment("Hola la",
>                new String[0])));
>    } catch (Throwable e) {
>        fail(msgNoException+e);
>    }
> }
>
> and
>
> public void
> testActivationGroupDescStringStringMarshalledObjectPropertiesCommandEnvironment007()
> {
>    try{
>        Properties p= new Properties();
>        assertNotNull(msgNotNull, new ActivationGroupDesc(null , null ,
>                new MarshalledObject(new Double(23.4))  ,  new Properties()
> ,
>                new ActivationGroupDesc.CommandEnvironment("", null)));
>    } catch (Throwable e) {
>        fail(msgNoException+e);
>    }
> }
>
>
> This is how the constructor under test looks like:
> public ActivationGroupDesc(String className, String codebase,
>        MarshalledObject data, Properties props,
>        ActivationGroupDesc.CommandEnvironment env) {
>
>    this.className = className;
>    this.location = codebase;
>    this.data = data;
>    this.props = props;
>    this.env = env;
> }
>
> It seems that instead of those million test cases we need just a few
> that would verify that getXXX() methods return what was passed into
> constructor
> plus possibly some tests that pass 'suspicious' parameters like null.
>
> Thoughts?
>
> Thanks,
> Mikhail
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message