harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sian January" <sianjanu...@googlemail.com>
Subject Re: [classlib][testing] Size of test input files
Date Thu, 17 Jan 2008 09:51:14 GMT
Hi Tim,

On 16/01/2008, Tim Ellison <t.p.ellison@gmail.com> wrote:

> Sian January wrote:
> > It's not really about using something of a certain size, it's more about
> > making sure we're testing with as wide a range as possible of different
> > bytecode combinations, class file attributes and things like inner
> classes
> > and java 5 features etc.
>
> Where do you envisage finding these packed archives to put into the test
> suite?



At the moment I am using Harmony modules, but I'm assuming that apart from
the space issue it would be ok to use any Apache licenced code.



> > Also if I generate source material I'm only
> > testing things I've thought of (which is more likely to work anyway
> since I
> > wrote the code).  Using large real-life programs tests things that I
> haven't
> > thought of and should also mean that the test coverage will be much
> higher.
>
> I can imagine you doing this locally to test the code, then generating
> your own examples to put in the test suite to get good coverage.


Yes - it would make sense to do some testing locally and not contribute it
all, especially if the code base isn't going to change that much.  I do
think it's still of value to have some real-life examples in the test suite
though, so I suppose I'm just trying to get a feel for how much is
appropriate.


> While you will be writing these and testing the code you wrote, that's
> not much different to you choosing the set of external archives to
> include then ensuring your code passes with them -- i.e. you will ensure
> your code works with the set you choose, which won't be exhaustive either.


I disagree here.  I think a week (or month etc) testing with different
existing code would produce much better coverage and a larger variety of
test cases than spending the same amount of time writing a code generator
would.  However I'll admit I don't know that much about generating code, so
maybe I'm missing something here...


> Regards,
> Tim
>

Thanks,

Sian

-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message