flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chiwan Park <chiwanp...@apache.org>
Subject Re: Tests - Unit Tests versus Integration Tests
Date Sat, 19 Sep 2015 16:33:57 GMT
Okay, I’ll create a JIRA issue and send a pull request for it. :)

Regards,
Chiwan Park

> On Sep 19, 2015, at 7:35 PM, Ufuk Celebi <uce@apache.org> wrote:
> 
> Thanks Stephan for pointing this out. I agree with you. +1
> 
> @Chiwan: Good idea with the Wiki. Actually maybe even better to add it to the contribution
guide? Do you have time to open a PR with Stephan’s suggestions?
> 
> @Martin: I agree that it does not suffice to just point this out as a new guideline.
I think the main problem is that it is very time consuming and error prone. We have seen some
“minor refactorings” lately, which looked harmless, but actually introduced bugs. This
is a danger as well with refactoring tests (we refactor them, but don’t have the same amount
of test coverage, which results in bugs at some point in time in the future).
> 
> Are there any known “heavy hitters”, which take a lot of time, but which could be
tested in unit tests instead? I would start with those if you want to do it. But in general
I would do this incrementally instead of aiming for a complete rewrite.
> 
> – Ufuk
> 
>> On 19 Sep 2015, at 10:53, Martin Liesenberg <martin.liesenberg@gmail.com> wrote:
>> 
>> Should there be a concerted effort to reduce the amount of unnecessary
>> integration tests and cover those cases by unit tests?
>> 
>> We could collect the cases in a ticket and work through the list one by
>> one, no?
>> 
>> Best regards,
>> Martin
>> 
>> Chiwan Park <chiwanpark@apache.org> schrieb am Fr., 18. Sep. 2015 um
>> 12:33 Uhr:
>> 
>>> Hi Stephan,
>>> 
>>> Thanks for nice guide! I think we can upload this to the wiki or how to
>>> contribute documentation.
>>> This guide would be helpful for newcomers.
>>> 
>>> Regards,
>>> Chiwan Park
>>> 
>>>> On Sep 17, 2015, at 9:33 PM, Stephan Ewen <sewen@apache.org> wrote:
>>>> 
>>>> Hi all!
>>>> 
>>>> The build time of Flink with all tests is nearing 1h on Travis for the
>>>> shortest run.
>>>> It is good that we do excessive testing, there are many mechanisms that
>>>> need that.
>>>> 
>>>> I have also seen that a lot of fixes that could be tested in a UnitTest
>>>> style are actually tested as a full Flink program (Integration test
>>> style)
>>>> 
>>>> While these tests are always easier to write, they have two problems:
>>>> - The bring up the build time by about 5 secs per test
>>>> - They are often not as targeted to the problem as a UnitTest
>>>> 
>>>> I would like to encourage everyone to keep this in mind and do Unit tests
>>>> in the cases where they are the preferred choice. Please also keep that
>>> in
>>>> mind when reviewing pull requests.
>>>> 
>>>> For Example:
>>>> - API / TypeInformation changes can be very well tested without running
>>>> the program. Simply create the program and test the operator's type info.
>>>> - Custom functions can be very well tested in isolation
>>>> - Input/Output formats actually test well in UnitTests.
>>>> 
>>>> Integration tests need to be used when verifying behavior across
>>> components
>>>> / layers, so keep using them when they need to be used.
>>>> 
>>>> 
>>>> Greetings,
>>>> Stephan
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
> 





Mime
View raw message