maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Connolly <stephen.alan.conno...@gmail.com>
Subject Re: Cobertura and Surefire
Date Thu, 13 Jan 2011 10:07:58 GMT
On 13 January 2011 09:50, Stefan Schulze <algroth@gmx.de> wrote:
> Stephen Connolly wrote:
>> Because people who have not read and understood concurrency in
>> practice often do not understand how the synchronization points affect
>> jvm sequencing, people often wrongly suspect that result of
>> instrumented and non-instrumented code is the same.
>
> I'm not sure, why caring about concurrency is important in this context. I'm not using
Maven 3 (so no concurrent builds) and I'm not using concurrent tests with JUnit.

Between synchronization points, the JVM is allowed to re-order
operations as long as the end result remains the same.

Cobertura puts a synchronization point in between every line and branch point.

Thus the re-ordering of lines cannot take place any more.

Start up a JVM and have a look at how many threads are running....

Still think you are in a single threaded world?

Also some simple libraries use threads behind your back... and even
the Java Runtime Libraries do that too...

Still think you don't have to know/worry about concurrency?

I shall repeat...

We had some "appears to be simple single threaded code" that we were
testing with a "simple test case".  The tests were only being run
instrumented... the tests were passing.... the bug was there... seen
in production system by big customer... in the end of the day, this
was identified as the original author not understanding about the
JVM's contract w.r.t. re-ordering operations in-between
synchronization points and the fact that using cobertura introduced a
synchronization point at every line so that the code as written could
only work when instrumented.

Run the damn tests at least twice. (I'd like to say run them on
multiple JVMs too but Oracle seems intent on merging all the JVMs into
openjdk)

-Stephen

> But it's quite plain to me it is possible, that instrumented code behaves different to
uninstrumented code, so testing the instrumented code can pass the tests while uninstrumented
fail. (i.e. because of attributes added by instrumentation, which are unexpected when working
with reflection on these attributes)
>
> But I think it doesn't happen on a regular basis, that the instrumented code pass or
fails the tests, where the uninstrumented doesn't. But sadly it's more regular in our team,
that the Cobertura-step fails (not because failing tests but because of too low coverage).
>

It happens when you least expect it.

>
> Wayne Fay wrote:
>> Perhaps bind something to your scm commit hook so the code
>> simply can't make it into your SCM tool without meeting
>> certain standards for coverage?
>
> That would mean, that during the commit we should run all the tests? I'm not sure, I
want the commit to last about two minutes.
> Beside this, we have to use Synergy/CM as SCM tool, which is not capable of pre-commit-hooks
(as far as I know).
>
> Hilco Wijbenga wrote:
>> A normal unit test should run once, more is just a waste of
>> time. The fact that Maven will run unit tests multiple times
>> is a bug that we apparently have to live with. It is not a feature.
>>
>> If this bug were fixed and you still wanted to run your unit
>> tests twice then you could configure that quite easily in your POM.
>> Everybody happy
>
> I don't really like that Maven needs to run tests twice (or more often), too. But I see
it is neccessary regarding Cobertura, because -- as stated to Stephen -- it can't be guaranteed,
that the instrumented code behaves the same as the uninstrumented. Because of this it is not
sufficient to run the tests only on the instrumented code.
> So this is not a bug in Maven but a problem with Cobertura.
> --
> GMX DSL Doppel-Flat ab 19,99 Euro/mtl.! Jetzt mit
> gratis Handy-Flat! http://portal.gmx.net/de/go/dsl
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
> For additional commands, e-mail: users-help@maven.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org


Mime
View raw message