db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tiago Espinha <ti...@espinha.net>
Subject Re: GSoC 2012 - To potential student applicants on the list
Date Fri, 13 Apr 2012 08:50:22 GMT
Hmm, if there's code covered by insane builds which is not covered by sane
builds, is that really a problem? We would just have to make sure that
regressions are ran against insane jars and all would be well. Or am I
missing something?

Yesterday Siddharth was also trying to obtain the Emma reports and he was
obtaining different results from those of the automated tests. It turns out
he was indeed running the tests against sane jars and he was running
derbyall instead of suites.All. If I'm not mistaken, suites.All is the one
that picks up all the JUnit tests whereas derbyall runs the old harness
tests. I was actually surprised that derbyall produced results at all...

Anyhow Nufail, well done! Be sure to include this in your proposal by April
16 at the latest as this is the hard deadline of when we will be evaluating
the proposals. I would also take up on Kathey's suggestion and run it with
different versions of JVMs. As far as I can see, you simply have to do two
separate test runs *in separate folders* using a different JVM for each
test run (different JVM = different JVM version). Then you will have two
coverage.em files which you should include when generating the report - at
that step you can include as many coverage files are you like and in this
case they are complementary to each other.

On Fri, Apr 13, 2012 at 1:27 AM, Mike Matrigali <mikem_app@sbcglobal.net>wrote:

> Katherine Marsden wrote:
>
>> On 4/12/2012 11:16 AM, Mohamed Nufail wrote:
>>
>>> Hi Tiago,
>>>
>>> I didn't run Emma coverage reports myself by the time I wrote the
>>> proposal. But I managed to do it today. It took a long time to run all the
>>> tests, but it completed without errors and produced a coverage report. The
>>> numbers were almost the same as that of the automated runs. I will update
>>> the proposal to include this.
>>>
>> Great work.  On thing I recall is that the person who used to run this
>> could run with multiple JVM's and get cumulative results.  For example
>> running both JDK 1.5 and JDK 1.6 would better cover both ClientDataSource
>> and ClientDataSource40.   If any of your targeted classes code path's
>> appear to be java version dependent,  it might be worthwhile to add JDK
>> 1.5, JDK 1.6 and JDK 1.7 in your baseline to see if they are already
>> covered with other java versions.
>>
>>
>>
>>
>>  I believe the automated tests are run against "insane" builds.  I think
> for identified missed codepaths getting a run against "sane" builds would
> be more valuable as I found that I kept tracking down uncovered
> links and they would lead to sanity code that is likely covered when
> running the tests in sane mode.  That with kathey's suggestion of
> adding runs against multiple jvms should good.  Next step would be
> to post it somewhere public so people can help identify what to look
> at and see progress.
>

Mime
View raw message