Hmm, if there's code covered by insane builds which is not covered by sane builds, is that really a problem? We would just have to make sure that regressions are ran against insane jars and all would be well. Or am I missing something?

Yesterday Siddharth was also trying to obtain the Emma reports and he was obtaining different results from those of the automated tests. It turns out he was indeed running the tests against sane jars and he was running derbyall instead of suites.All. If I'm not mistaken, suites.All is the one that picks up all the JUnit tests whereas derbyall runs the old harness tests. I was actually surprised that derbyall produced results at all...

Anyhow Nufail, well done! Be sure to include this in your proposal by April 16 at the latest as this is the hard deadline of when we will be evaluating the proposals. I would also take up on Kathey's suggestion and run it with different versions of JVMs. As far as I can see, you simply have to do two separate test runs *in separate folders* using a different JVM for each test run (different JVM = different JVM version). Then you will have two coverage.em files which you should include when generating the report - at that step you can include as many coverage files are you like and in this case they are complementary to each other.

On Fri, Apr 13, 2012 at 1:27 AM, Mike Matrigali <mikem_app@sbcglobal.net> wrote:
Katherine Marsden wrote:
On 4/12/2012 11:16 AM, Mohamed Nufail wrote:
Hi Tiago,

I didn't run Emma coverage reports myself by the time I wrote the proposal. But I managed to do it today. It took a long time to run all the tests, but it completed without errors and produced a coverage report. The numbers were almost the same as that of the automated runs. I will update the proposal to include this.
Great work.  On thing I recall is that the person who used to run this could run with multiple JVM's and get cumulative results.  For example running both JDK 1.5 and JDK 1.6 would better cover both ClientDataSource and ClientDataSource40.   If any of your targeted classes code path's appear to be java version dependent,  it might be worthwhile to add JDK 1.5, JDK 1.6 and JDK 1.7 in your baseline to see if they are already covered with other java versions.




I believe the automated tests are run against "insane" builds.  I think
for identified missed codepaths getting a run against "sane" builds would be more valuable as I found that I kept tracking down uncovered
links and they would lead to sanity code that is likely covered when
running the tests in sane mode.  That with kathey's suggestion of
adding runs against multiple jvms should good.  Next step would be
to post it somewhere public so people can help identify what to look
at and see progress.