xmlgraphics-batik-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Bowditch <bowditch_ch...@hotmail.com>
Subject Re: Batik Test Suite
Date Fri, 08 Aug 2014 08:07:54 GMT
Hi Vincent,

Thanks for the update. Ideally we should investigate the remaining 
failures before the release

Thanks,

Chris

On 07/08/2014 19:53, Vincent Hennebert wrote:
> Some update on this.
>
> More than half of the failing tests (369 exactly), actually correspond
> to slight rendering differences between the reference image and what my
> platform produces (a lot of them due to different fonts being used, but
> not all).
>
> I could update the reference image, but then other people on other
> platforms may still get different results.
>
> I prefer to use the available ‘accepted variation’ mechanism: Batik
> generates a difference image between the reference and the actual. I can
> decide that the difference is small enough that it’s acceptable, so
> I copy the png from the candidate-variation folder to
> accepted-variation.
>
> I can even add a platform-specific suffix if I wish to
> (_java6-linux in my case, the other one available is _java5-osx which
> probably is outdated). That would be helpful for people who work on
> various different platforms, but I found out that not all tests offer
> that possibility, so in the end it’s of limited use.
>
> I attached to this message the list of tests that can safely be marked
> as accepted.
>
> There are 180 remaining tests that fail. I put the report on my public
> space on Apache:
> http://people.apache.org/~vhennebert/batik-test-suite/testReport/html/regardReport.html

>
>
> Most of them belong to one of the 8 following categories:
> • TranscoderException
> • NullPointerException
> • Gradients not being rendered
> • Roundtrip failing: generating a memory representation of the SVG and 
> rendering it back to an SVG file
> • Permissions issues
> • Memory leaks
> • WMF conversion issues
> • Some tests that don't honour an accepted-variation
>
> So the next step would be to investigate what’s going wrong in each
> category and fix the issue.
>
> But that’s another story...
>
> Vincent
>
>
> On 28/07/14 23:43, Vincent Hennebert wrote:
>> Hi,
>>
>> I thought I would post here my findings about running the Batik test
>> suite.
>>
>> The test suite can be run on the command line by using the ‘regard’
>> target from build.xml. It produces an HTML report that lists the failing
>> test cases.
>>
>> To begin with, the Basic Effectivity test suite from the SVG Working
>> Group will probably have to be removed. That’s because it’s not shipped
>> with Batik (the code is pointing to a directory outside the project
>> root), and I can no longer find any trace of it on the Internet. Unless
>> I missed something, of course.
>>
>> The rest of the test suite runs but results into many failures (nearly
>> 500). Most of the tests probably just need to be updated (some of them
>> are based on the comparison of a bitmap rendering against a reference
>> image). But some of them may well point at regressions and I can see
>> quite a few exceptions. Do we want to take the risk of releasing Batik
>> with a failing test suite? Probably not.
>>
>> This test suite looks like it was created before JUnit became the de
>> facto standard for testing. It looks rather powerful and elegantly
>> designed but, needless to say, it doesn’t quite have the same tooling
>> support as JUnit (IDE integration, CI, code coverage, etc.). Also, it
>> opens up several windows to render some SVG tests, which is quite
>> disrupting. We might want to convert it to JUnit and reduce it to
>> a subset that can run in the background.
>>
>> But the most urgent probably is to go through the test results and
>> investigate the failing ones.
>>
>> Vincent
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
> For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: batik-dev-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-dev-help@xmlgraphics.apache.org


Mime
View raw message