flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Examples of Flink windowing unit tests, and option to bring in flink-spector to Flink
Date Thu, 04 May 2017 00:52:03 GMT
See this related thread:

http://search-hadoop.com/m/Flink/VkLeQcgMO91NWi982?subj=flink+1+2+and+unit+testing+flinkspector

On Wed, May 3, 2017 at 3:20 AM, Aljoscha Krettek <aljoscha@apache.org>
wrote:

> Hi,
> Yes, unfortunately there are no good unit test examples for testing Flink
> jobs (with windowing or without). And yes, I would like to have something
> like flink-spector (or flink-spector) as part of Flink as the promoted way
> of doing Job testing. In my opinion this is blocked on a good way of
> verifying that assertions succeed inside operators and on an API to
> programmatically monitor and cancel a running streaming Job.
>
> If you had those two components (asynchronous job management API and
> assertion verification) you could do something similar to PAssert (
> https://beam.apache.org/documentation/sdks/javadoc/0.
> 6.0/index.html?org/apache/beam/sdk/testing/PAssert.html <
> https://beam.apache.org/documentation/sdks/javadoc/0.
> 6.0/index.html?org/apache/beam/sdk/testing/PAssert.html>) in Apache Beam
> that is used for a lot of tests. The idea is that assertion operators
> inside the pipeline wait for a condition to be true (or fail of some
> negative condition is violated) and signal success by updating a unique
> Accumulator value. The test would use the asynchronous job API to listen on
> Accumulator updates and wait until all required conditions have been met.
> Then it would stop the job and the test would succeed.
>
> Best,
> Aljoscha
>
> > On 2. May 2017, at 22:01, Bowen Li <bowen.li@offerupnow.com> wrote:
> >
> > Hi Flink folks!
> >     Are there any concrete good examples of unit testing Flink windowing?
> > I've been searching around, and all those available online are quite
> > ambiguous.
> >
> >     Another thing is, I found the flink-spector
> > <https://github.com/ottogroup/flink-spector> library is pretty awesome,
> > simple to understand and use. But it's kind of out of maintenance, and
> only
> > supports 1.4 now. Is it possible to bring flink-spector into Apache
> Flink,
> > so we can better upgrade and maintain it?
> >
> > Thank you!
> > Bowen
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message