brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Duncan Grant <duncan.gr...@cloudsoftcorp.com>
Subject Re: Code coverage with cobertura
Date Thu, 15 Jan 2015 13:53:05 GMT
Aled,

Here's the top-line coverage for each project for unit tests

api/target/site/cobertura/coverage.xml                                 :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="523" branches-covered="0" branches-valid="130"
complexity="1.14520202020202" version="1.9.4.1" timestamp="1421328275796">
camp/camp-base/target/site/cobertura/coverage.xml                      :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328287176">
camp/camp-server/target/site/cobertura/coverage.xml                    :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328298297">
core/target/site/cobertura/coverage.xml                                :
<coverage line-rate="0.640080146231721" branch-rate="0.5358428549068712"
lines-covered="18209" lines-valid="28448" branches-covered="6847"
branches-valid="12778" complexity="2.3311258278145695" version="1.9.4.1"
timestamp="1421328399007">
examples/global-web-fabric/target/site/cobertura/coverage.xml          :
<coverage line-rate="0.0" branch-rate="1.0" lines-covered="0"
lines-valid="40" branches-covered="0" branches-valid="0" complexity="1.0"
version="1.9.4.1" timestamp="1421328859307">
examples/simple-messaging-pubsub/target/site/cobertura/coverage.xml    :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="87" branches-covered="0" branches-valid="6"
complexity="1.8333333333333333" version="1.9.4.1" timestamp="1421328864312">
examples/simple-nosql-cluster/target/site/cobertura/coverage.xml       :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="257" branches-covered="0" branches-valid="10"
complexity="1.3043478260869565" version="1.9.4.1" timestamp="1421328869077">
examples/simple-web-cluster/target/site/cobertura/coverage.xml         :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="177" branches-covered="0" branches-valid="2"
complexity="1.1111111111111112" version="1.9.4.1" timestamp="1421328854477">
examples/webapps/hello-world-sql/target/site/cobertura/coverage.xml    :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328847126">
examples/webapps/hello-world-webapp/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328845013">
locations/jclouds/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.21518987341772153" branch-rate="0.13455882352941176"
lines-covered="629" lines-valid="2923" branches-covered="183"
branches-valid="1360" complexity="2.8916155419222904" version="1.9.4.1"
timestamp="1421328479641">
policy/target/site/cobertura/coverage.xml                              :
<coverage line-rate="0.8294375456537618" branch-rate="0.6083202511773941"
lines-covered="2271" lines-valid="2738" branches-covered="775"
branches-valid="1274" complexity="2.4641350210970465" version="1.9.4.1"
timestamp="1421328452571">
software/base/target/site/cobertura/coverage.xml                       :
<coverage line-rate="0.45638406537282944" branch-rate="0.35834333733493395"
lines-covered="2234" lines-valid="4895" branches-covered="597"
branches-valid="1666" complexity="2.194888178913738" version="1.9.4.1"
timestamp="1421328526470">
software/database/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="721" branches-covered="0" branches-valid="138"
complexity="1.4970059880239521" version="1.9.4.1" timestamp="1421328585506">
software/messaging/target/site/cobertura/coverage.xml                  :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="1081" branches-covered="0" branches-valid="148"
complexity="1.2607449856733524" version="1.9.4.1" timestamp="1421328577885">
software/monitoring/target/site/cobertura/coverage.xml                 :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="99" branches-covered="0" branches-valid="12"
complexity="1.3043478260869565" version="1.9.4.1" timestamp="1421328608506">
software/network/target/site/cobertura/coverage.xml                    :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="265" branches-covered="0" branches-valid="52"
complexity="1.4166666666666667" version="1.9.4.1" timestamp="1421328534584">
software/nosql/target/site/cobertura/coverage.xml                      :
<coverage line-rate="0.16019818331957061" branch-rate="0.21768707482993196"
lines-covered="582" lines-valid="3633" branches-covered="256"
branches-valid="1176" complexity="2.0057887120115776" version="1.9.4.1"
timestamp="1421328600464">
software/osgi/target/site/cobertura/coverage.xml                       :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="181" branches-covered="0" branches-valid="34" complexity="1.6"
version="1.9.4.1" timestamp="1421328539923">
software/webapp/target/site/cobertura/coverage.xml                     :
<coverage line-rate="0.23116883116883116" branch-rate="0.21767241379310345"
lines-covered="623" lines-valid="2695" branches-covered="202"
branches-valid="928" complexity="2.0690298507462686" version="1.9.4.1"
timestamp="1421328568254">
storage/hazelcast/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="47" branches-covered="0" branches-valid="4"
complexity="1.2352941176470589" version="1.9.4.1" timestamp="1421328459030">
usage/all/target/site/cobertura/coverage.xml                           :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328811306">
usage/archetypes/quickstart/target/site/cobertura/coverage.xml         :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328842718">
usage/camp/target/site/cobertura/coverage.xml                          :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328650960">
usage/cli/target/site/cobertura/coverage.xml                           :
<coverage line-rate="0.33574007220216606" branch-rate="0.22580645161290322"
lines-covered="279" lines-valid="831" branches-covered="70"
branches-valid="310" complexity="3.3636363636363638" version="1.9.4.1"
timestamp="1421328802377">
usage/dist/target/site/cobertura/coverage.xml                          :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328827806">
usage/jsgui/target/site/cobertura/coverage.xml                         :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328743328">
usage/launcher/target/site/cobertura/coverage.xml                      :
<coverage line-rate="0.5664516129032258" branch-rate="0.49158249158249157"
lines-covered="439" lines-valid="775" branches-covered="146"
branches-valid="297" complexity="2.7540983606557377" version="1.9.4.1"
timestamp="1421328788133">
usage/logback-includes/target/site/cobertura/coverage.xml              :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="11" branches-covered="0" branches-valid="2" complexity="1.2"
version="1.9.4.1" timestamp="1421328251510">
usage/logback-xml/target/site/cobertura/coverage.xml                   :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421328292376">
usage/qa/target/site/cobertura/coverage.xml                            :
<coverage line-rate="0.12598425196850394" branch-rate="0.13157894736842105"
lines-covered="80" lines-valid="635" branches-covered="25"
branches-valid="190" complexity="2.5301204819277108" version="1.9.4.1"
timestamp="1421328876342">
usage/rest-api/target/site/cobertura/coverage.xml                      :
<coverage line-rate="0.18016759776536312" branch-rate="0.16417910447761194"
lines-covered="129" lines-valid="716" branches-covered="66"
branches-valid="402" complexity="1.659025787965616" version="1.9.4.1"
timestamp="1421328656401">
usage/rest-client/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.14285714285714285" branch-rate="0.0"
lines-covered="4" lines-valid="28" branches-covered="0" branches-valid="4"
complexity="1.2105263157894737" version="1.9.4.1" timestamp="1421328722761">
usage/rest-server/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.6810602821718683" branch-rate="0.4912959381044487"
lines-covered="1593" lines-valid="2339" branches-covered="508"
branches-valid="1034" complexity="3.0214477211796247" version="1.9.4.1"
timestamp="1421328710813">
usage/test-support/target/site/cobertura/coverage.xml                  :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="614" branches-covered="0" branches-valid="306"
complexity="1.8611111111111112" version="1.9.4.1" timestamp="1421328282364">
utils/common/target/site/cobertura/coverage.xml                        :
<coverage line-rate="0.5037804076265615" branch-rate="0.4570335491465568"
lines-covered="3065" lines-valid="6084" branches-covered="1553"
branches-valid="3398" complexity="2.4800979791794244" version="1.9.4.1"
timestamp="1421328262980">
utils/groovy/target/site/cobertura/coverage.xml                        :
<coverage line-rate="0.3781094527363184" branch-rate="0.30434782608695654"
lines-covered="76" lines-valid="201" branches-covered="84"
branches-valid="276" complexity="1.0" version="1.9.4.1"
timestamp="1421328272060">
utils/jmx/jmxmp-ssl-agent/target/site/cobertura/coverage.xml           :
<coverage line-rate="0.5338345864661654" branch-rate="0.38636363636363635"
lines-covered="71" lines-valid="133" branches-covered="17"
branches-valid="44" complexity="3.1052631578947367" version="1.9.4.1"
timestamp="1421328490860">
utils/jmx/jmxrmi-agent/target/site/cobertura/coverage.xml              :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="56" branches-covered="0" branches-valid="14" complexity="2.4"
version="1.9.4.1" timestamp="1421328494007">
utils/rest-swagger/target/site/cobertura/coverage.xml                  :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="126" branches-covered="0" branches-valid="86" complexity="3.7"
version="1.9.4.1" timestamp="1421328290376">
utils/test-support/target/site/cobertura/coverage.xml                  :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="290" branches-covered="0" branches-valid="108"
complexity="2.204081632653061" version="1.9.4.1" timestamp="1421328248357">

And here are the partial numbers for integration tests:

api/target/site/cobertura/coverage.xml                                 :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="523" branches-covered="0" branches-valid="130"
complexity="1.14520202020202" version="1.9.4.1" timestamp="1421255489048">
camp/camp-base/target/site/cobertura/coverage.xml                      :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421255496879">
camp/camp-server/target/site/cobertura/coverage.xml                    :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421255504839">
examples/global-web-fabric/target/site/cobertura/coverage.xml          :
<coverage line-rate="0.0" branch-rate="1.0" lines-covered="0"
lines-valid="40" branches-covered="0" branches-valid="0" complexity="1.0"
version="1.9.4.1" timestamp="1421262294641">
examples/simple-messaging-pubsub/target/site/cobertura/coverage.xml    :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="87" branches-covered="0" branches-valid="6"
complexity="1.8333333333333333" version="1.9.4.1" timestamp="1421262300045">
examples/simple-nosql-cluster/target/site/cobertura/coverage.xml       :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="257" branches-covered="0" branches-valid="10"
complexity="1.3043478260869565" version="1.9.4.1" timestamp="1421262304724">
examples/simple-web-cluster/target/site/cobertura/coverage.xml         :
<coverage line-rate="0.2937853107344633" branch-rate="0.5"
lines-covered="52" lines-valid="177" branches-covered="1"
branches-valid="2" complexity="1.1111111111111112" version="1.9.4.1"
timestamp="1421262290268">
examples/webapps/hello-world-sql/target/site/cobertura/coverage.xml    :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262239392">
examples/webapps/hello-world-webapp/target/site/cobertura/coverage.xml :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262237740">
locations/jclouds/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.10708176530961341" branch-rate="0.03823529411764706"
lines-covered="313" lines-valid="2923" branches-covered="52"
branches-valid="1360" complexity="2.8916155419222904" version="1.9.4.1"
timestamp="1421256566766">
policy/target/site/cobertura/coverage.xml                              :
<coverage line-rate="0.5467494521548576" branch-rate="0.35949764521193095"
lines-covered="1497" lines-valid="2738" branches-covered="458"
branches-valid="1274" complexity="2.4641350210970465" version="1.9.4.1"
timestamp="1421256487045">
software/network/target/site/cobertura/coverage.xml                    :
<coverage line-rate="0.6679245283018868" branch-rate="0.6153846153846154"
lines-covered="177" lines-valid="265" branches-covered="32"
branches-valid="52" complexity="1.4166666666666667" version="1.9.4.1"
timestamp="1421257862576">
software/osgi/target/site/cobertura/coverage.xml                       :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="181" branches-covered="0" branches-valid="34" complexity="1.6"
version="1.9.4.1" timestamp="1421257867961">
storage/hazelcast/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.5319148936170213" branch-rate="0.25"
lines-covered="25" lines-valid="47" branches-covered="1" branches-valid="4"
complexity="1.2352941176470589" version="1.9.4.1" timestamp="1421256552341">
usage/archetypes/quickstart/target/site/cobertura/coverage.xml         :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262235938">
usage/cli/target/site/cobertura/coverage.xml                           :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="831" branches-covered="0" branches-valid="310"
complexity="3.3636363636363638" version="1.9.4.1" timestamp="1421262219150">
usage/jsgui/target/site/cobertura/coverage.xml                         :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421262131024">
usage/launcher/target/site/cobertura/coverage.xml                      :
<coverage line-rate="0.5690322580645162" branch-rate="0.43097643097643096"
lines-covered="441" lines-valid="775" branches-covered="128"
branches-valid="297" complexity="2.7540983606557377" version="1.9.4.1"
timestamp="1421262212380">
usage/logback-includes/target/site/cobertura/coverage.xml              :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="11" branches-covered="0" branches-valid="2" complexity="1.2"
version="1.9.4.1" timestamp="1421255459645">
usage/logback-xml/target/site/cobertura/coverage.xml                   :
<coverage line-rate="1.0" branch-rate="1.0" lines-covered="0"
lines-valid="0" branches-covered="0" branches-valid="0" complexity="0.0"
version="1.9.4.1" timestamp="1421255500917">
usage/qa/target/site/cobertura/coverage.xml                            :
<coverage line-rate="0.08346456692913386" branch-rate="0.07368421052631578"
lines-covered="53" lines-valid="635" branches-covered="14"
branches-valid="190" complexity="2.5301204819277108" version="1.9.4.1"
timestamp="1421262312249">
usage/rest-api/target/site/cobertura/coverage.xml                      :
<coverage line-rate="0.06284916201117319"
branch-rate="0.024875621890547265" lines-covered="45" lines-valid="716"
branches-covered="10" branches-valid="402" complexity="1.659025787965616"
version="1.9.4.1" timestamp="1421262082264">
usage/rest-client/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.25" branch-rate="0.0" lines-covered="7"
lines-valid="28" branches-covered="0" branches-valid="4"
complexity="1.2105263157894737" version="1.9.4.1" timestamp="1421262115976">
usage/rest-server/target/site/cobertura/coverage.xml                   :
<coverage line-rate="0.36896109448482256" branch-rate="0.23404255319148937"
lines-covered="863" lines-valid="2339" branches-covered="242"
branches-valid="1034" complexity="3.0214477211796247" version="1.9.4.1"
timestamp="1421262103160">
usage/test-support/target/site/cobertura/coverage.xml                  :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="614" branches-covered="0" branches-valid="306"
complexity="1.8611111111111112" version="1.9.4.1" timestamp="1421255492975">
utils/common/target/site/cobertura/coverage.xml                        :
<coverage line-rate="0.14896415652745806" branch-rate="0.08387286639199529"
lines-covered="906" lines-valid="6082" branches-covered="285"
branches-valid="3398" complexity="2.481912936848559" version="1.9.4.1"
timestamp="1421255479993">
utils/groovy/target/site/cobertura/coverage.xml                        :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="201" branches-covered="0" branches-valid="276"
complexity="1.0" version="1.9.4.1" timestamp="1421255486003">
utils/jmx/jmxmp-ssl-agent/target/site/cobertura/coverage.xml           :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="133" branches-covered="0" branches-valid="44"
complexity="3.1052631578947367" version="1.9.4.1" timestamp="1421256573037">
utils/jmx/jmxrmi-agent/target/site/cobertura/coverage.xml              :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="56" branches-covered="0" branches-valid="14" complexity="2.4"
version="1.9.4.1" timestamp="1421256575531">
utils/rest-swagger/target/site/cobertura/coverage.xml                  :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="126" branches-covered="0" branches-valid="86" complexity="3.7"
version="1.9.4.1" timestamp="1421255499592">
utils/test-support/target/site/cobertura/coverage.xml                  :
<coverage line-rate="0.0" branch-rate="0.0" lines-covered="0"
lines-valid="290" branches-covered="0" branches-valid="108"
complexity="2.204081632653061" version="1.9.4.1" timestamp="1421255457257">

We can see that we expect the coverage to go up when we add integration
tests however these numbers represent code covered by integration tests in
the same project (I think) so aren't as high as they should be.

On 15 January 2015 at 12:17, Aled Sage <aled.sage@gmail.com> wrote:

> Hi Duncan,
>
> Excellent, code coverage reports will be very useful moving forwards.
>
> ---
> Regarding generating a single report, here are some comments that were
> posted off-list:
>
>    Could you accumulate the reports as you go? Rather than running one
>    merge at the end do it for each project after the
>    post-integration-test or package phase, perhaps. I can imagine how
>    you would configure this, can't think how you'd ask for something to
>    be run at the end of the build.
>
>    Sam
>
> And:
>
>    Have you tried the aggregate parameter -
>    http://mojo.codehaus.org/cobertura-maven-plugin/
> cobertura-mojo.html#aggregate?
>    Here’s an example that should aggregate it’s results:
>    https://svn.codehaus.org/mojo/trunk/mojo/cobertura-maven-
> plugin/src/it/mcobertura-65/pom.xml
>
>    Svet.
>
> ---
> For xml versus html, If it's easy to say both then might as well do that.
> Otherwise, the html report will be more useful in the short term.
>
> ---
> For integration tests, sounds like they need some tender loving care
> again. I'll try to find some time to look at any failures, but it might not
> be until next week. If anyone else picks up fixing any of these, then
> please do ping me if you want help.
>
> ---
> For what numbers to expect, I'm hoping for good test coverage in core,
> policies and utils. For the projects that contain mostly entities, I expect
> poor unit test coverage but reasonably good integration test coverage. The
> reason is that running the entities installs software etc, so in many
> places it is hard to unit test that beyond the not-particularly-useful what
> commands would it have wanted to execute.
>
> Aled
>
>
>
> On 14/01/2015 10:26, Duncan Grant wrote:
>
>> I'm trying to get code coverage working for brooklyn.
>>
>> As a first task I've upgraded the version of cobertura we are using and
>> fixed some issues with paths.
>> This means that you can run mvn install -P Coverage and it will produce
>> coverage reports for individual projects.  I'll create a PR today.
>>
>> I've tried swapping the ant-run approach that is currently used for just
>> the maven plugin but there were some errors that I couldn't resolve and I
>> don't think it's worth the effort.
>>
>> I'd also like to have a single report for coverage across the project.  I
>> think I just need a task that runs after all the projects have been built
>> but I'm not sure how to do that as the parent project seems to run first.
>> I haven't much experience with maven so I'm probably missing something
>> obvious.
>>
>> Does anyone have an opinion on whether we should be producing xml reports
>> or html reports or both?  The html report is a nice human readable report.
>> The xml report can be integrated (using a plugin) with jenkins to show
>> trends but I'm not sure how helpful that is.
>>
>> I'm also a bit stuck getting the integration tests to run (due to a failed
>> dependancy over night).  Also it seems to have been a while since they
>> complete successfully on http://brooklyn.builds.cloudsoftcorp.com/ .  So
>> I
>> don't have any idea what coverage will look like then.
>>
>> Any thoughts would be appreciated
>>
>> Duncan
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message