beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stas Levin (JIRA)" <>
Subject [jira] [Commented] (BEAM-1399) Code coverage numbers are not accurate
Date Sun, 19 Feb 2017 09:42:44 GMT


Stas Levin commented on BEAM-1399:

I wasn't aware the {{beam-sdks-java-javadoc}} module depended on all the other ones, thanks
for the tip!

>From preliminary experiments, looks like {{report-aggregate}} aggregates reports across
dependent modules, so that the resulting report is as if you had executed the regular {{report}}
and merged them all together (visually). 
In terms of coverage, I'm still seeing the 69% for {{beam-sdks-java-core}} both when using
{{report}} specifically for {{beam-sdks-java-core}} and {{report-aggregate}} for the {{beam-sdks-java-javadoc}}
module. This may indicate that {{report-aggregate}} does not capture the additional {{@RunnableOnService/@NeedsRunner}}
tests we are after, at least not out-of-the box. 
In addition, test class coverage is not reported (neither by {{report}} nor by {{report-aggregate}}),
only production code ^2^.

Essentially we have the following major issues:
# Record coverage across modules 
#* Might be possible using {{maven-antrun-plugin}} and/or copying {{.class}} / {{.java}} files
of dependent modules to {{beam-sdks-java-javadoc}}, as described in ^1^.
# Report test class coverage 
#* Might be possible using {{maven-antrun-plugin}} since it gives fine-grained control over
that's reported.
# Merge coverage provided by different executions of the same test class 
#* As I dive deeper into this, I'm inclined to think that the {{jacoco-maven-plugin}} does
not support merging coverage for a test class executed from different modules due to ^3^,
which brings me back to separating {{@RunnableOnService/@NeedsRunner}} into their own test
classes and the rest of the goodies behind curtain number (2) detailed in my first comment


> Code coverage numbers are not accurate
> --------------------------------------
>                 Key: BEAM-1399
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: build-system, sdk-java-core, testing
>            Reporter: Daniel Halperin
>              Labels: newbie, starter
> We've started adding Java Code Coverage numbers to PRs using the jacoco plugin. However,
we are getting very low coverage reported for things like the Java SDK core.
> My belief is that this is happening because we test the bulk of the SDK not in the SDK
module , but in fact in the DirectRunner and other similar modules.
> JaCoCo has a {{report:aggregate}} target that might do the trick, but with a few minutes
of playing with it I wasn't able to make it work satisfactorily. Basic work in
> This is a good "random improvement" issue for anyone to pick up.

This message was sent by Atlassian JIRA

View raw message