harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Ellison <t.p.elli...@gmail.com>
Subject Re: [classlib] testing guidlines -- support classes
Date Wed, 05 Jul 2006 11:28:36 GMT
I think that we are really struggling here to represent the different
combination of tests using the file system directory.

So far I think we have :

 - api tests on application classpath
 - impl tests on bootclasspath
 - impl tests on application classpath
 - support classes
 - platform specific classes
 - failing tests (even temp failing and always failing)
 - resources used in tests

and I'm sure we will find more categories as we go.

IMHO trying to use a combination of directory and file names to separate
these is flawed.  Each new categorization adds another layer in source
tree layout, and added complexity.

I still believe that we should be using suites to organize the tests
into groups, irrespective of how they are stored on disk.  In this case,
categorization is by suite membership, not based on file path.

Glad I got that off my chest.

Regards,
Tim


Mikhail Loenko wrote:
> 2006/7/3, Tim Ellison <t.p.ellison@gmail.com>:
>> Mikhail Loenko wrote:
>> > All: support classes, impl-classpath, impl-boot, api-classpath, and
>> > api-boot tests
>> > are compiled into separate directories.
>>
>> ( TestNG looking better by the hour eh? ;-) )
>>
>> > Support classes are compiled first, tests are compiled with support
>> > classes in
>> > the classpath.
>>
>> Presumably you don't mind if they are all compiled together?  Just
>> figuring out how to set up my IDE descriptions.
>>
>> > When the tests are running, support classes are accessible the same
>> way as
>> > the tests, i.e. via bootclasspath when bootclasspath tests are running
>> > and in
>> > classpath when classpath tests are running.
>>
>> Ok.
>>
>> > If nobody object I can add to the doc something like:
>> >
>> > "Support classes for the tests might be located both within the tests
>> > and in the
>> > support directory. Support classes that are shared between different
>> > types of the
>> > tests are recommended to be in the support dir.
>>
>> It may be worth describing what you mean by a 'support class'.
> 
> Ok
> 
>>
>> > When the bootclasspath tests are running all classes in the support
>> > dir are available by bootclasspath. Otherwise they are available by
>> > classpath."
>> > via classpath.
>>
>> A simple example would help too.
> 
> not sure what you mean...
> 
> Thanks,
> Mikhail
> 
>>
>> Regards,
>> Tim
>>
>> > 2006/7/3, Tim Ellison <t.p.ellison@gmail.com>:
>> >> Mikhail Loenko wrote:
>> >> > That means that all the API tests will be in the bootclasspath when
>> >> > impl/bootclasspath tests run? Will this run be clear enough?
>> >>
>> >> No I don't think that will be clear.
>> >>
>> >> So can you describe how the bootclasspath and classpath are set up for
>> >> running each set of tests?  Maybe we should add that info to the
>> >> document?
>> >>
>> >> Also I'm left wondering how the support classes should be compiled?
>> >> Presumably they can be compiled against the APIs only?
>> >>
>> >> Regards,
>> >> Tim
>> >>
>> >>
>> >> > 2006/6/30, Tim Ellison <t.p.ellison@gmail.com>:
>> >> >> Mikhail Loenko wrote:
>> >> >> > I'm refereing to those support classes that are used by both
API
>> >> >> > and impl tests
>> >> >>
>> >> >> Sure, but if the support classes themselves only use API then
>> they can
>> >> >> be in the api dir right?  i.e. we expect our impl tests to use
some
>> >> APIs
>> >> >> too.
>> >> >>
>> >> >> Regards,
>> >> >> Tim
>> >> >>
>> >> >> > 2006/6/30, Tim Ellison <t.p.ellison@gmail.com>:
>> >> >> >> Mikhail Loenko wrote:
>> >> >> >> > There are support classes that are shared by various
types
>> of the
>> >> >> >> tests,
>> >> >> >> > e.g. api and impl or classpath and bootclasspath
>> >> >> >> >
>> >> >> >> > We can either separate them from the tests or duplicate.
>> >> >> >>
>> >> >> >> I'm confused.
>> >> >> >>
>> >> >> >> Either the support classes are used by API tests and only
>> make API
>> >> >> calls
>> >> >> >> into the code, or they are harmony implementation-specific;
and
>> >> either
>> >> >> >> they need to be loaded on the bootclasspath or not --
right?
>> >> >> >>
>> >> >> >> So can't they just go into the existing bin directory
>> >> classifications?
>> >> >> >>
>> >> >> >> Regards,
>> >> >> >> Tim
>> >> >> >>
>> >> >> >> > I suggest that we separate these classes into folder
>> >> >> >> >    src/test/support/[platform]/java
>> >> >> >> > and
>> >> >> >> >    org.apache.harmony.module.tests.support.*
>> >> >> >> > package.
>> >> >> >> >
>> >> >> >> > Probably it makes sense to put all the module's support
>> >> >> >> > classes there (not only shared ones).
>> >> >> >> >
>> >> >> >> > Thoughts?
>> >> >> >> >
>> >> >> >> > Thanks,
>> >> >> >> > Mikhail
>> >> >> >> >
>> >> >> >> >
>> >> >>
>> ---------------------------------------------------------------------
>> >> >> >> > Terms of use :
>> http://incubator.apache.org/harmony/mailing.html
>> >> >> >> > To unsubscribe, e-mail:
>> >> harmony-dev-unsubscribe@incubator.apache.org
>> >> >> >> > For additional commands, e-mail:
>> >> >> harmony-dev-help@incubator.apache.org
>> >> >> >> >
>> >> >> >> >
>> >> >> >>
>> >> >> >> --
>> >> >> >>
>> >> >> >> Tim Ellison (t.p.ellison@gmail.com)
>> >> >> >> IBM Java technology centre, UK.
>> >> >> >>
>> >> >> >>
>> >> ---------------------------------------------------------------------
>> >> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >> >> To unsubscribe, e-mail:
>> >> harmony-dev-unsubscribe@incubator.apache.org
>> >> >> >> For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >> >>
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> ---------------------------------------------------------------------
>> >> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >> > To unsubscribe, e-mail:
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> >> > For additional commands, e-mail:
>> >> harmony-dev-help@incubator.apache.org
>> >> >> >
>> >> >> >
>> >> >>
>> >> >> --
>> >> >>
>> >> >> Tim Ellison (t.p.ellison@gmail.com)
>> >> >> IBM Java technology centre, UK.
>> >> >>
>> >> >>
>> ---------------------------------------------------------------------
>> >> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> >> To unsubscribe, e-mail:
>> harmony-dev-unsubscribe@incubator.apache.org
>> >> >> For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>> >> >>
>> >> >>
>> >> >
>> >> >
>> ---------------------------------------------------------------------
>> >> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> > For additional commands, e-mail:
>> harmony-dev-help@incubator.apache.org
>> >> >
>> >> >
>> >>
>> >> --
>> >>
>> >> Tim Ellison (t.p.ellison@gmail.com)
>> >> IBM Java technology centre, UK.
>> >>
>> >> ---------------------------------------------------------------------
>> >> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> >> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>> >>
>> >>
>> >
>> > ---------------------------------------------------------------------
>> > Terms of use : http://incubator.apache.org/harmony/mailing.html
>> > To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> > For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>> >
>> >
>>
>> -- 
>>
>> Tim Ellison (t.p.ellison@gmail.com)
>> IBM Java technology centre, UK.
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
> 
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
> 
> 

-- 

Tim Ellison (t.p.ellison@gmail.com)
IBM Java technology centre, UK.

---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message