Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\))
Subject: Re: Tests - Unit Tests versus Integration Tests
From: Chiwan Park <chiwanpark@apache.org>
In-Reply-To: <DF347C0D-1B10-4343-AD15-474E96322CDA@apache.org>
Date: Sun, 20 Sep 2015 01:33:57 +0900
Content-Transfer-Encoding: quoted-printable
Message-Id: <93803DE5-020C-435D-A9F9-397A2F42ABD2@apache.org>
References: 
 <CANC1h_vvekciNVDzqCb8N4E5Kfzu4e1Mosnse1=V11HXnD2PBQ@mail.gmail.com>
 <4C320A4A-D2EA-422D-97B4-798E47DA3F12@apache.org>
 <CAF60_64rjhrj80F13++A_Jp297qx5gq3PkF32srmfhUhrRerJg@mail.gmail.com>
 <DF347C0D-1B10-4343-AD15-474E96322CDA@apache.org>
To: dev@flink.apache.org

Okay, I=E2=80=99ll create a JIRA issue and send a pull request for it. =
:)

Regards,
Chiwan Park

> On Sep 19, 2015, at 7:35 PM, Ufuk Celebi <uce@apache.org> wrote:
>=20
> Thanks Stephan for pointing this out. I agree with you. +1
>=20
> @Chiwan: Good idea with the Wiki. Actually maybe even better to add it =
to the contribution guide? Do you have time to open a PR with =
Stephan=E2=80=99s suggestions?
>=20
> @Martin: I agree that it does not suffice to just point this out as a =
new guideline. I think the main problem is that it is very time =
consuming and error prone. We have seen some =E2=80=9Cminor =
refactorings=E2=80=9D lately, which looked harmless, but actually =
introduced bugs. This is a danger as well with refactoring tests (we =
refactor them, but don=E2=80=99t have the same amount of test coverage, =
which results in bugs at some point in time in the future).
>=20
> Are there any known =E2=80=9Cheavy hitters=E2=80=9D, which take a lot =
of time, but which could be tested in unit tests instead? I would start =
with those if you want to do it. But in general I would do this =
incrementally instead of aiming for a complete rewrite.
>=20
> =E2=80=93 Ufuk
>=20
>> On 19 Sep 2015, at 10:53, Martin Liesenberg =
<martin.liesenberg@gmail.com> wrote:
>>=20
>> Should there be a concerted effort to reduce the amount of =
unnecessary
>> integration tests and cover those cases by unit tests?
>>=20
>> We could collect the cases in a ticket and work through the list one =
by
>> one, no?
>>=20
>> Best regards,
>> Martin
>>=20
>> Chiwan Park <chiwanpark@apache.org> schrieb am Fr., 18. Sep. 2015 um
>> 12:33 Uhr:
>>=20
>>> Hi Stephan,
>>>=20
>>> Thanks for nice guide! I think we can upload this to the wiki or how =
to
>>> contribute documentation.
>>> This guide would be helpful for newcomers.
>>>=20
>>> Regards,
>>> Chiwan Park
>>>=20
>>>> On Sep 17, 2015, at 9:33 PM, Stephan Ewen <sewen@apache.org> wrote:
>>>>=20
>>>> Hi all!
>>>>=20
>>>> The build time of Flink with all tests is nearing 1h on Travis for =
the
>>>> shortest run.
>>>> It is good that we do excessive testing, there are many mechanisms =
that
>>>> need that.
>>>>=20
>>>> I have also seen that a lot of fixes that could be tested in a =
UnitTest
>>>> style are actually tested as a full Flink program (Integration test
>>> style)
>>>>=20
>>>> While these tests are always easier to write, they have two =
problems:
>>>> - The bring up the build time by about 5 secs per test
>>>> - They are often not as targeted to the problem as a UnitTest
>>>>=20
>>>> I would like to encourage everyone to keep this in mind and do Unit =
tests
>>>> in the cases where they are the preferred choice. Please also keep =
that
>>> in
>>>> mind when reviewing pull requests.
>>>>=20
>>>> For Example:
>>>> - API / TypeInformation changes can be very well tested without =
running
>>>> the program. Simply create the program and test the operator's type =
info.
>>>> - Custom functions can be very well tested in isolation
>>>> - Input/Output formats actually test well in UnitTests.
>>>>=20
>>>> Integration tests need to be used when verifying behavior across
>>> components
>>>> / layers, so keep using them when they need to be used.
>>>>=20
>>>>=20
>>>> Greetings,
>>>> Stephan
>>>=20
>>>=20
>>>=20
>>>=20
>>>=20
>>>=20
>=20