flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chiwan Park <chiwanp...@apache.org>
Subject Re: Tests - Unit Tests versus Integration Tests
Date Sat, 19 Sep 2015 16:39:48 GMT
I just created a JIRA issue [1].

Regards,
Chiwan Park

[1] https://issues.apache.org/jira/browse/FLINK-2712

> On Sep 20, 2015, at 1:33 AM, Chiwan Park <chiwanpark@apache.org> wrote:
> 
> Okay, I’ll create a JIRA issue and send a pull request for it. :)
> 
> Regards,
> Chiwan Park
> 
>> On Sep 19, 2015, at 7:35 PM, Ufuk Celebi <uce@apache.org> wrote:
>> 
>> Thanks Stephan for pointing this out. I agree with you. +1
>> 
>> @Chiwan: Good idea with the Wiki. Actually maybe even better to add it to the contribution
guide? Do you have time to open a PR with Stephan’s suggestions?
>> 
>> @Martin: I agree that it does not suffice to just point this out as a new guideline.
I think the main problem is that it is very time consuming and error prone. We have seen some
“minor refactorings” lately, which looked harmless, but actually introduced bugs. This
is a danger as well with refactoring tests (we refactor them, but don’t have the same amount
of test coverage, which results in bugs at some point in time in the future).
>> 
>> Are there any known “heavy hitters”, which take a lot of time, but which could
be tested in unit tests instead? I would start with those if you want to do it. But in general
I would do this incrementally instead of aiming for a complete rewrite.
>> 
>> – Ufuk
>> 
>>> On 19 Sep 2015, at 10:53, Martin Liesenberg <martin.liesenberg@gmail.com>
wrote:
>>> 
>>> Should there be a concerted effort to reduce the amount of unnecessary
>>> integration tests and cover those cases by unit tests?
>>> 
>>> We could collect the cases in a ticket and work through the list one by
>>> one, no?
>>> 
>>> Best regards,
>>> Martin
>>> 
>>> Chiwan Park <chiwanpark@apache.org> schrieb am Fr., 18. Sep. 2015 um
>>> 12:33 Uhr:
>>> 
>>>> Hi Stephan,
>>>> 
>>>> Thanks for nice guide! I think we can upload this to the wiki or how to
>>>> contribute documentation.
>>>> This guide would be helpful for newcomers.
>>>> 
>>>> Regards,
>>>> Chiwan Park
>>>> 
>>>>> On Sep 17, 2015, at 9:33 PM, Stephan Ewen <sewen@apache.org> wrote:
>>>>> 
>>>>> Hi all!
>>>>> 
>>>>> The build time of Flink with all tests is nearing 1h on Travis for the
>>>>> shortest run.
>>>>> It is good that we do excessive testing, there are many mechanisms that
>>>>> need that.
>>>>> 
>>>>> I have also seen that a lot of fixes that could be tested in a UnitTest
>>>>> style are actually tested as a full Flink program (Integration test
>>>> style)
>>>>> 
>>>>> While these tests are always easier to write, they have two problems:
>>>>> - The bring up the build time by about 5 secs per test
>>>>> - They are often not as targeted to the problem as a UnitTest
>>>>> 
>>>>> I would like to encourage everyone to keep this in mind and do Unit tests
>>>>> in the cases where they are the preferred choice. Please also keep that
>>>> in
>>>>> mind when reviewing pull requests.
>>>>> 
>>>>> For Example:
>>>>> - API / TypeInformation changes can be very well tested without running
>>>>> the program. Simply create the program and test the operator's type info.
>>>>> - Custom functions can be very well tested in isolation
>>>>> - Input/Output formats actually test well in UnitTests.
>>>>> 
>>>>> Integration tests need to be used when verifying behavior across
>>>> components
>>>>> / layers, so keep using them when they need to be used.
>>>>> 
>>>>> 
>>>>> Greetings,
>>>>> Stephan
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>> 
> 
> 
> 
> 

Mime
View raw message