Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: pass (nike.apache.org: local policy)
Date: Thu, 21 Feb 2008 16:43:21 +0100
From: Vemund Ostgaard <Vemund.Ostgaard@Sun.COM>
Subject: Re: Using EMMA for codecoverage analysis
In-reply-to: <47BC636E.4090806@apache.org>
Sender: Vemund.Ostgaard@Sun.COM
To: derby-dev@db.apache.org
Message-id: <47BD9C19.3080405@sun.com>
MIME-version: 1.0
Content-type: text/plain; format=flowed; charset=ISO-8859-1
Content-transfer-encoding: 7BIT
References: <47AC7399.6070602@sun.com> <47BC398E.9040301@sun.com>
 <47BC636E.4090806@apache.org>
User-Agent: Thunderbird 2.0.0.9 (X11/20071119)

Daniel John Debrunner wrote:
> Vemund Ostgaard wrote:
>> I got suites.All to run with EMMA, but I had to add 7-8 lines to 
>> derby_tests.policy and modify a couple of tests before it ran cleanly.
>>
>> I think it would be good if this worked without editing the source, 
>> so I'll make a Jira and upload a patch with these changes.
>
> It might be good to separate out the test changes, I'd be interested 
> to understand why they needed to be changed to run under code coverage.
I will do that. I've only fixed suites.All, but I saw similar cases in 
the derbyall test as well. I think the emma instrumentation affects the 
tests in two ways:

1. Running instrumented code requires emma.jar in the classpath. If a 
test forks a new process that runs instrumented code (for instance ij) 
the new process also needs emma.jar in the classpath. The same is true 
if a test creates a new classloader that loads an instrumented class, 
emma.jar is needed in that classloaders search URL as well.

2. Code instrumented by EMMA will by default write some output, which 
some tests don't like. Using -Demma.verbosity.level=silent when starting 
junit solves this for most tests, but if a test forks a new process it 
may have to set the emma.verbosity.level for that process as well if it 
is touchy about the extra output.

It was only two tests in suites.All that needed modifications, but I 
guess we might see these two issues also in the future when new tests 
are written.
>
>> I'm thinking about adding some ant tasks for instrumenting 
>> classes/jars and running the junit-tests with EMMA, making it easier 
>> to do this.
>
> +1
I hadn't run the junit tests with ant before and had some problems 
running them with ant 1.6.5, so I switched to ant 1.7 and it is working 
better.

I'm now working my way through the extra permissions needed for 
junit.jar and ant-junit.jar when running with instrumented jarfiles. I 
also think some permissions are needed when running with ant 1.7 that 
wasn't needed when running with ant 1.6.5.

Vemund