flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: Published test artifacts for flink streaming
Date Fri, 20 Nov 2015 19:31:40 GMT
Very interesting Alex!

One other thing I find useful in building data flows is using "builder"
functions that hide the details of wiring up specific plumbing on generic
input parameters. For instance a void wireFoo(DataSource source,
SinkFunction sink) { ... }. It would be great to have test tools that allow
working with this kind of composition. For instance, I find invoking
setParallelism on the result of registering a SinkFunction on a flow is not
very convenient with the current APIs. In this case, I would want to set
the parallelism to something different in test than I'd like in production.

Any thoughts on how we might make testing compositions like this easier?


On Thu, Nov 19, 2015 at 2:09 AM, lofifnc <alexander.kolb@mni.thm.de> wrote:

> Hi,
> I'm currently working on improving the testing process of flink streaming
> applications.
> I have written a test runtime that takes care of execution, collecting the
> output and applying a verifier to it. The runtime is able to provide test
> sources and sinks that run in parallel.
> On top of that it offers an API for defining JUnit tests. I have recreated
> your test using my API:
> https://gist.github.com/anonymous/8825ea83e8fe9afba19e
> I have a second example that shows some of the more advanced features of
> the
> API and runtime:
> https://gist.github.com/anonymous/fa5c77becae2e37d28eb
> Unfortunately It's still a work in progress. But I'm planning to have a
> first release candidate at the end of the month.
> Best, Alex
> --
> View this message in context:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Published-test-artifacts-for-flink-streaming-tp3379p3584.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.

View raw message