ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jaikiran Pai <jai.forums2...@gmail.com>
Subject Re: Ant support for users who use JUnit5
Date Thu, 15 Feb 2018 13:51:46 GMT
I'm now done with the initial goals that I had in mind for this task. 
I've opened a PR[1] for review. I've included a manual for this task and 
it can be currently found here[2]. I would suggest reading the manual 
first, before reviewing the PR, since the manual will give an overall 
idea of what's being attempted with this task.

Except for the "fork" mode which is available in our existing junit task 
and which was was in the TODO list, in my previous mail, the rest have 
been implemented in this version. I plan to focus on the "fork" mode 
after this task is made available to users and any feedback received.

I have tried to address some of the (internal) technicalities, based on 
some discussions that I have seen about JUnit, XML reporter and such, 
are around OOMs issues when it comes to sysout and syserr handling. 
Result formatters, which can be configured as listeners, are allowed to 
say "sendSysOut" and/or "sendSysErr". In such cases, the task will 
(internally) redirect the sysout/syserr to a PipedOutputStream (which is 
backed by a PipedInputStream). The task also sets up 2 separate threads 
- one which reads the PipedInputStream and "feeds" it to the second 
thread which "delivers" this content to such result formatters. The 
necessity to use a separate thread to deliver the content of 
sysout/syserr is so that we don't run into deadlocks (as noted in the 
Javadoc of Piped(output/input)Stream class) if/when the result 
formatters themselves write something out to sysout/syserr when they are 
working on this incoming sysout/syserr content.

We have 3 pre-defined formatters all of which are capable of receiving 
this streamed sysout/syserr data. Each of these do _not_ hold on to this 
sysout/syserr data in-memory during the execution and instead stream it 
out a temporary file. Once the execution completes and it's time to 
present the result, these formatters stream/read back the content from 
the temp file and write it out in a formatted manner to the target 
report files.

Furthermore, the XML formatter doesn't use DOM and instead is based on 
Stax for writing out the report. However, even with all this, there's 
one place where I haven't yet been able to avoid reading the whole 
generated sysout/syserr data into memory (thus potentially triggering an 
OOM) - it's the XMLStreamWriter API which for writing out a CDATA 
section (which is what we use for sysout/syserr content) in XML, expects 
the entire String object. I'll have to see if there are ways to avoid 
it, but I think this still is an improvement since this loaded content 
is only held on for a short while in memory, during the report writing 
and will be immediately garbage collected once that's done.

Please review the PR - mostly the Ant specific constructs and 
implementation details.

[1] https://github.com/apache/ant/pull/60

[2] 
https://builds.apache.org/job/Ant-Build-Jaikiran/ws/manual/Tasks/junitlauncher.html

[3] 
https://github.com/apache/ant/pull/60/files#diff-f985f89ce4779eff2cdcf164d09d5396R308

-Jaikiran




On 27/01/18 8:40 AM, Jaikiran Pai wrote:
> Here's an update on where this effort now stands. As of yesterday, I 
> have the basic minimal functionality that I had planned for this (new) 
> task ready. There are other enhancements that this task will need as 
> we go along but at this point, this should be usable.
>
> While working on this new task, I kept thinking whether it would make 
> more sense to just have this task as a separate project under the Ant 
> umbrella and have its own release/versioning cycle. Plus maybe a bit 
> of ease in building/testing it given that it won't need some of the 
> conditional logic that we do in Ant build itself when it comes to such 
> tasks. It did look like a good idea to separate it out but ultimately 
> I decided _not_ to do it mainly for the reason that I think it makes 
> more sense to have a task in Ant which allows users to write tests 
> (using a modern test framework) and test them, right out of the box of 
> an Ant installation. This first class experience, IMO, outweighs all 
> the other "advantages" that seem to come with having this as a 
> separate project.
>
> Coming to the task itself, the task is called "junitlauncher". In the 
> current state of this task[1], the following goals/features are 
> implemented and functional:
>
> - Ability to launch the JUnit 5 platform
>
> - Ability to specify a classpath to use for the launched tests
>
> - Ability to specify one or more single test classes that need to be 
> run as part of this launch
>
> - Ability to specify specific methods on these test classses that need 
> to be run as part of the launch
>
> - Ability to specify "batch tests" which essentially is a way to use 
> Ant's resource collections to pattern match files that should be 
> passed on to the JUnit platform to be evaluated and run as test cases.
>
> - Ability to specify "listeners" for tests. These listeners are 
> expected to implement (intentionally) JUnit platform's interface and 
> _not_ any of Ant's interfaces.
>
> - Test result formatters are implemented as "listeners". This task 
> comes with 2 (for now) implementations out of the box, "plain" and 
> "brief". The idea behind these 2 is the same as that of what we 
> current have with the "junit" task formatters.
>
> - Users can define custom formatters as "listeners" by specifying a 
> class which implements the JUnit's test listener plus (optionally) one 
> of Ant's own (new) custom interface. This custom interface will let 
> them have access to output stream to which they might want to write 
> out the results.
>
> (I might be missing a few more details, but these are the major 
> functional features. The manual that I plan to write, will have the 
> whole details)
>
> In its current state the task should be able to run both "vintage" 
> (JUnit 4.x) and "jupiter" (JUnit 5.x) based tests.
>
> TODO/enhancements for later (a few days/weeks down the line):
>
> - Ability to "fork" these tests in a separate JVM. I haven't fully 
> thought about this and might need some inputs on whether we need this 
> or not. If we do add this, I might do it slightly differently that 
> what we current do with "junit" where there are numerous attributes to 
> the task/test elements which are only applicable if fork mode is 
> enabled. I might perhaps just introduce a new element within the 
> task's element which specifically is meant to deal with any forked VM 
> characteristics. But that's something I will get to after the other 
> easier enhancements are done.
>
> - Provide the XML formatter out of the box. I was planning to do this 
> in the first version itself, but I haven't had enough time to 
> understand the schema of this XML plus whether or not the details that 
> we put in here are available through the new JUnit launcher APIs.
>
> - Ability to more specifically say which JUnit test engine needs to be 
> used for the tests. Right now, the classpath decides which engine gets 
> used and thus which classes are considered as tests. If the classpath 
> has both "vintage" and "jupiter" engines then such tests will be run. 
> However, I want users to be able to say "just use jupiter engine for 
> these tests" without having to worry about checking if the classpath 
> is polluted with some other engines.
>
> - Make the current "junitreport" task be usable with the XML results 
> of this "junitlauncher" task. I haven't yet got to this but this 
> should be doable I think, of course once the XML formatter itself is 
> ready and functional. I don't plan to introduce a new task for this 
> and instead plan to reuse/enhance the existing junitreport task to 
> work seamlessly both with existing "junit" task and the 
> "junitlauncher" task.
>
> (Few other minor enhancements here and there and any user feedback 
> reports)
>
> For those of you curious to see what the task usage is going to look 
> like, here's an example[2] build file which shows its many usages. 
> That file will be cleaned up a bit, before I send a PR for review/merge.
>
> [1] https://github.com/apache/ant/compare/master...jaikiran:junit5
>
> [2] 
> https://github.com/apache/ant/compare/master...jaikiran:junit5#diff-3e0911ed925129e68c543aca0abc5d7aR52
>
> -Jaikiran
>
>
> On 14/12/17 5:14 PM, Stefan Bodewig wrote:
>> On 2017-12-14, Jaikiran Pai wrote:
>>
>>> With that context, I would like to explain what I have attempted so
>>> far and I would like inputs on whether I should follow this path
>>> and/or if there are other suggestions on how we should go ahead. As
>>> you are probably aware, the JUnitTask (the Ant task backing the
>>> <junit> element) has very complex logic which deals with launching the
>>> test runners responsible for running JUnit tests (3.x or 4.x). IMO,
>>> most this logic can now be handed off to the JUnit 5 platform
>>> launcher, using their APIs.
>> In addition some of the complexity inside of JUnitTask stems from the
>> fact that it must not refer to any JUnit 4.x API in order to be usable
>> in a pure JUnit 3.x environment. If you try to add JUnit 5 on top of
>> that this sounds diffciult.
>>
>> I too had toyed with the idea of adding JUnit5 support but it is quite a
>> bit further down my TODO list and I've not looked into JUnit5 as deeply
>> as you did. What you've got so far looks promising, please please push
>> ahead.
>>
>> My idea was to create a completely new task and mainly try to keep the
>> XML output the way it has been for some fifteen years so junitreport
>> would still work. I even though about placing it in a separate antlib
>> rather than Ant's core to allow it get released independently.
>>
>> My vote would go to a separate task.
>>
>> Stefan
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
>> For additional commands, e-mail: dev-help@ant.apache.org
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message