Mailing-List: contact dev-help@harmony.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@harmony.apache.org
Received-SPF: pass (athena.apache.org: domain of sianjanuary@googlemail.com
 designates 209.85.146.178 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=googlemail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references;
        b=Qd3/tYNd4jiTeQGF04bVjpOy+9h9lDeAVxrqCsxFbFgMBj0qtieKnBKCltWfo7JkJ5IQxCii3TB8cbVfeRXtr9YcvETah12f95ugFPc3UgT8jtoPMOqXHl3wa9otuAzY8K47QkOhiocFRdKw5tPpl/ugtCFTQ5HJqYemk4t5HWI=
Message-ID: <fc2fc95c0801170151m498c21d9hf14d6ca33be9a16b@mail.gmail.com>
Date: Thu, 17 Jan 2008 09:51:14 +0000
From: "Sian January" <sianjanuary@googlemail.com>
To: dev@harmony.apache.org
Subject: Re: [classlib][testing] Size of test input files
In-Reply-To: <478E41A2.7000807@gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_37426_11544525.1200563474919"
References: <fc2fc95c0801160304uac9ed15wb7d8eefa0e273304@mail.gmail.com>
	 <6e47b64f0801160315k723954d5q71de650b2bc52428@mail.gmail.com>
	 <fc2fc95c0801160324w2cbef92dpb51bce9eac6ac90f@mail.gmail.com>
	 <6e47b64f0801160329s23ba1ce4geb830b56408aa2f9@mail.gmail.com>
	 <fc2fc95c0801160347j44053079x99d361f575d0269c@mail.gmail.com>
	 <478E41A2.7000807@gmail.com>

------=_Part_37426_11544525.1200563474919
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi Tim,

On 16/01/2008, Tim Ellison <t.p.ellison@gmail.com> wrote:

> Sian January wrote:
> > It's not really about using something of a certain size, it's more about
> > making sure we're testing with as wide a range as possible of different
> > bytecode combinations, class file attributes and things like inner
> classes
> > and java 5 features etc.
>
> Where do you envisage finding these packed archives to put into the test
> suite?


At the moment I am using Harmony modules, but I'm assuming that apart from
the space issue it would be ok to use any Apache licenced code.


> > Also if I generate source material I'm only
> > testing things I've thought of (which is more likely to work anyway
> since I
> > wrote the code).  Using large real-life programs tests things that I
> haven't
> > thought of and should also mean that the test coverage will be much
> higher.
>
> I can imagine you doing this locally to test the code, then generating
> your own examples to put in the test suite to get good coverage.


Yes - it would make sense to do some testing locally and not contribute it
all, especially if the code base isn't going to change that much.  I do
think it's still of value to have some real-life examples in the test suite
though, so I suppose I'm just trying to get a feel for how much is
appropriate.


> While you will be writing these and testing the code you wrote, that's
> not much different to you choosing the set of external archives to
> include then ensuring your code passes with them -- i.e. you will ensure
> your code works with the set you choose, which won't be exhaustive either.


I disagree here.  I think a week (or month etc) testing with different
existing code would produce much better coverage and a larger variety of
test cases than spending the same amount of time writing a code generator
would.  However I'll admit I don't know that much about generating code, so
maybe I'm missing something here...


> Regards,
> Tim
>

Thanks,

Sian

-- 
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

------=_Part_37426_11544525.1200563474919--