flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Richter <s.rich...@data-artisans.com>
Subject Re: Unit/Integration test for stateful source
Date Thu, 20 Sep 2018 14:43:55 GMT
Hi,

maybe you can use AbstractStreamOperatorTestHarness to test your source, including the snapshotting.
You can take a look at the tests of some other source, e.g. StatefulSequenceSourceTest#testCheckpointRestore.

Best,
Stefan

> Am 20.09.2018 um 15:29 schrieb Darshan Singh <darshan.meel@gmail.com>:
> 
> Hi,
> 
> I am writing a stateful source very similar to KafkaBaseConsumer but not as generic.
 I was looking on how we can use unit test cases and integration tests on this. I looked at
the kafka-connector-based unit test cases. It seems that there is too much external things
at play here like lots of mock classes created and then even Abstract fetcher was created.

> 
> So I sort of dropped the idea where I can check if snapshot is basically calling offset
clear as well as store the offsets. 
> 
> But doing the integration tests is hard as well especially as I want to see if it picks
up offsets properly. I can run it manually and see it but I was hoping if we can get some
directions on integration tests for stateful source.
> 
> Thanks


Mime
View raw message