flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Examples of Flink windowing unit tests, and option to bring in flink-spector to Flink
Date Wed, 03 May 2017 10:20:35 GMT
Hi,
Yes, unfortunately there are no good unit test examples for testing Flink jobs (with windowing
or without). And yes, I would like to have something like flink-spector (or flink-spector)
as part of Flink as the promoted way of doing Job testing. In my opinion this is blocked on
a good way of verifying that assertions succeed inside operators and on an API to programmatically
monitor and cancel a running streaming Job.

If you had those two components (asynchronous job management API and assertion verification)
you could do something similar to PAssert (https://beam.apache.org/documentation/sdks/javadoc/0.6.0/index.html?org/apache/beam/sdk/testing/PAssert.html
<https://beam.apache.org/documentation/sdks/javadoc/0.6.0/index.html?org/apache/beam/sdk/testing/PAssert.html>)
in Apache Beam that is used for a lot of tests. The idea is that assertion operators inside
the pipeline wait for a condition to be true (or fail of some negative condition is violated)
and signal success by updating a unique Accumulator value. The test would use the asynchronous
job API to listen on Accumulator updates and wait until all required conditions have been
met. Then it would stop the job and the test would succeed.

Best,
Aljoscha

> On 2. May 2017, at 22:01, Bowen Li <bowen.li@offerupnow.com> wrote:
> 
> Hi Flink folks!
>     Are there any concrete good examples of unit testing Flink windowing?
> I've been searching around, and all those available online are quite
> ambiguous.
> 
>     Another thing is, I found the flink-spector
> <https://github.com/ottogroup/flink-spector> library is pretty awesome,
> simple to understand and use. But it's kind of out of maintenance, and only
> supports 1.4 now. Is it possible to bring flink-spector into Apache Flink,
> so we can better upgrade and maintain it?
> 
> Thank you!
> Bowen


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message