maven-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stig Rohde Døssing (JIRA) <j...@apache.org>
Subject [jira] [Commented] (SUREFIRE-1226) Surefire trims all stack trace elements except those in the test that failed
Date Sun, 01 Oct 2017 14:01:07 GMT

    [ https://issues.apache.org/jira/browse/SUREFIRE-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187391#comment-16187391
] 

Stig Rohde Døssing commented on SUREFIRE-1226:
----------------------------------------------

Thanks for responding [~tibor17].

You identify these two requirements:
* Users want trimStackTrace to trim only lines before the test method is reached, so it is
possible to identify which line in a unit under test threw an exception.
* Users want trimStackTrace to trim lines before and after the test method is reached when
those lines are e.g. Hamcrest or AssertJ lines.

The documentation for trimStackTrace says it will "trim the stack trace in the reports to
just the lines within the test".

For versions 2.13-2.18.1, trimStackTrace supported the first requirement, but not the second,
by interpreting "lines within the test" to mean "lines deeper in the stack trace than the
test method".

2.19 changes the behavior so the first requirement is not supported, but the second is, by
interpreting "lines within the test" to mean "lines containing the location of the test".

I agree that trimStackTrace can't fulfill both requirements, but from 2.13-2.18.1 it fulfilled
requirement 1 and not 2, and now it fulfills requirement 2 but not 1. Even if the documentation
has been here since 2012, the interpretation of it has clearly changed.

I think both requirements (and interpretations of trimStackTrace) are equally valid, but the
pre-2.19 behavior is the least harmful because it errs on the side of printing too many lines
(e.g. the extra Hamcrest lines) rather than too few (the trimmed lines inside the unit under
test). 

We're trying to argue that switching to the 2.19 behavior is an odd tradeoff, because users
with requirement 2 are now saving a few trace lines compared to before, but users with requirement
1 end up with a misleading trace. Also keep in mind that users with requirement 1 is going
to be anyone who gets an unexpected exception in a test from outside the test method, so this
is likely to affect a lot of users.

There's also a visibility issue since trimStackTrace is on by default, and as this thread
shows at least a few people have run into trouble because the new behavior is unexpected.
The factor of surprise could be reduced by inserting lines in the trace notifying about deleted
lines, as [~jglick] suggests, even if reverting to the 2.18.1 behavior is not feasible. 

My preferred solution would be reverting to the 2.18.1 behavior. If this is not possible it
would be good to add the extra lines suggested by [~jglick] so it's at least visible what's
happening. 

The more flexible trimStackTrace in 3.0 sounds great. We will probably work around this issue
until then by disabling trimStackTrace.

> Surefire trims all stack trace elements except those in the test that failed
> ----------------------------------------------------------------------------
>
>                 Key: SUREFIRE-1226
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1226
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Maven Surefire Plugin
>    Affects Versions: 2.19
>            Reporter: Jiri Locker
>            Assignee: Tibor Digana
>            Priority: Minor
>         Attachments: mvn_stacktrace.tar.gz
>
>
> This looks like a re-occurrence of SUREFIRE-992 but now the stack trace is even more
trimmed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message