Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
Subject: Re: Tests - Unit Tests versus Integration Tests
From: Chiwan Park <chiwanpark@apache.org>
In-Reply-To: <93803DE5-020C-435D-A9F9-397A2F42ABD2@apache.org>
Date: Sun, 20 Sep 2015 01:39:48 +0900
Content-Transfer-Encoding: quoted-printable
Message-Id: <AE2D9BDE-E7CE-4B11-BA65-2679A2CDCD72@apache.org>
References: 
 <CANC1h_vvekciNVDzqCb8N4E5Kfzu4e1Mosnse1=V11HXnD2PBQ@mail.gmail.com>
 <4C320A4A-D2EA-422D-97B4-798E47DA3F12@apache.org>
 <CAF60_64rjhrj80F13++A_Jp297qx5gq3PkF32srmfhUhrRerJg@mail.gmail.com>
 <DF347C0D-1B10-4343-AD15-474E96322CDA@apache.org>
 <93803DE5-020C-435D-A9F9-397A2F42ABD2@apache.org>
To: dev@flink.apache.org

I just created a JIRA issue [1].

Regards,
Chiwan Park

[1] https://issues.apache.org/jira/browse/FLINK-2712

> On Sep 20, 2015, at 1:33 AM, Chiwan Park <chiwanpark@apache.org> =
wrote:
>=20
> Okay, I=E2=80=99ll create a JIRA issue and send a pull request for it. =
:)
>=20
> Regards,
> Chiwan Park
>=20
>> On Sep 19, 2015, at 7:35 PM, Ufuk Celebi <uce@apache.org> wrote:
>>=20
>> Thanks Stephan for pointing this out. I agree with you. +1
>>=20
>> @Chiwan: Good idea with the Wiki. Actually maybe even better to add =
it to the contribution guide? Do you have time to open a PR with =
Stephan=E2=80=99s suggestions?
>>=20
>> @Martin: I agree that it does not suffice to just point this out as a =
new guideline. I think the main problem is that it is very time =
consuming and error prone. We have seen some =E2=80=9Cminor =
refactorings=E2=80=9D lately, which looked harmless, but actually =
introduced bugs. This is a danger as well with refactoring tests (we =
refactor them, but don=E2=80=99t have the same amount of test coverage, =
which results in bugs at some point in time in the future).
>>=20
>> Are there any known =E2=80=9Cheavy hitters=E2=80=9D, which take a lot =
of time, but which could be tested in unit tests instead? I would start =
with those if you want to do it. But in general I would do this =
incrementally instead of aiming for a complete rewrite.
>>=20
>> =E2=80=93 Ufuk
>>=20
>>> On 19 Sep 2015, at 10:53, Martin Liesenberg =
<martin.liesenberg@gmail.com> wrote:
>>>=20
>>> Should there be a concerted effort to reduce the amount of =
unnecessary
>>> integration tests and cover those cases by unit tests?
>>>=20
>>> We could collect the cases in a ticket and work through the list one =
by
>>> one, no?
>>>=20
>>> Best regards,
>>> Martin
>>>=20
>>> Chiwan Park <chiwanpark@apache.org> schrieb am Fr., 18. Sep. 2015 um
>>> 12:33 Uhr:
>>>=20
>>>> Hi Stephan,
>>>>=20
>>>> Thanks for nice guide! I think we can upload this to the wiki or =
how to
>>>> contribute documentation.
>>>> This guide would be helpful for newcomers.
>>>>=20
>>>> Regards,
>>>> Chiwan Park
>>>>=20
>>>>> On Sep 17, 2015, at 9:33 PM, Stephan Ewen <sewen@apache.org> =
wrote:
>>>>>=20
>>>>> Hi all!
>>>>>=20
>>>>> The build time of Flink with all tests is nearing 1h on Travis for =
the
>>>>> shortest run.
>>>>> It is good that we do excessive testing, there are many mechanisms =
that
>>>>> need that.
>>>>>=20
>>>>> I have also seen that a lot of fixes that could be tested in a =
UnitTest
>>>>> style are actually tested as a full Flink program (Integration =
test
>>>> style)
>>>>>=20
>>>>> While these tests are always easier to write, they have two =
problems:
>>>>> - The bring up the build time by about 5 secs per test
>>>>> - They are often not as targeted to the problem as a UnitTest
>>>>>=20
>>>>> I would like to encourage everyone to keep this in mind and do =
Unit tests
>>>>> in the cases where they are the preferred choice. Please also keep =
that
>>>> in
>>>>> mind when reviewing pull requests.
>>>>>=20
>>>>> For Example:
>>>>> - API / TypeInformation changes can be very well tested without =
running
>>>>> the program. Simply create the program and test the operator's =
type info.
>>>>> - Custom functions can be very well tested in isolation
>>>>> - Input/Output formats actually test well in UnitTests.
>>>>>=20
>>>>> Integration tests need to be used when verifying behavior across
>>>> components
>>>>> / layers, so keep using them when they need to be used.
>>>>>=20
>>>>>=20
>>>>> Greetings,
>>>>> Stephan
>>>>=20
>>>>=20
>>>>=20
>>>>=20
>>>>=20
>>>>=20
>>=20
>=20
>=20
>=20
>=20