flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Published test artifacts for flink streaming
Date Thu, 05 Nov 2015 23:32:34 GMT
Hey!

There is also a collect() sink in the "flink-streaming-contrib" project,
see here:
https://github.com/apache/flink/blob/master/flink-contrib/flink-streaming-contrib/src/main/java/org/apache/flink/contrib/streaming/DataStreamUtils.java

It should work well locally for testing. In that case you can write a
program as usual an use "DataStreamUtils.collect(stream)", so you need to
stop reading it once you know the stream is exhausted...

Stephan


On Thu, Nov 5, 2015 at 10:38 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:

> Hi Robert,
>
> It seems "type" was what I needed. This it also looks like the test
> jar has an undeclared dependency. In the end, the following allowed me
> to use TestStreamEnvironment for my integration test. Thanks a lot!
>
> -n
>
>     <dependency>
>       <groupId>org.apache.flink</groupId>
>       <artifactId>flink-streaming-core</artifactId>
>       <version>${flink.version}</version>
>       <type>test-jar</type>
>       <scope>test</scope>
>     </dependency>
>     <dependency>
>       <groupId>org.apache.flink</groupId>
>       <artifactId>flink-test-utils</artifactId>
>       <version>${flink.version}</version>
>       <scope>test</scope>
>     </dependency>
>
> On Thu, Nov 5, 2015 at 12:47 PM, Robert Metzger <rmetzger@apache.org>
> wrote:
> > Hi Nick,
> >
> > we are usually publishing the test  artifacts. Can you try and replace
> the
> > <classifier> tag by <type>test-jar<type>:
> >
> > <dependency>
> >    <groupId>org.apache.flink</groupId>
> >    <artifactId>flink-streaming-core</artifactId>
> >    <version>${flink.version}</version>
> >    <type>test-jar</type>
> >    <scope>test</scope>
> > </dependency>
> >
> >
> > On Thu, Nov 5, 2015 at 8:18 PM, Nick Dimiduk <ndimiduk@gmail.com> wrote:
> >>
> >> Hello,
> >>
> >> I'm attempting integration tests for my streaming flows. I'd like to
> >> produce an input stream of java objects and sink the results into a
> >> collection for verification via JUnit asserts.
> >> StreamExecutionEnvironment provides methods for the former, however,
> >> how to achieve the latter is not evident based on my internet
> >> searching. I think I've found a solution in the TestStreamEnvironment
> >> class, ie, as used by WindowingIntegrationTest. However, this class
> >> appears to be packaged in the flink-streaming-core test artifact,
> >> which is not published to maven.
> >>
> >> For reference, this is the maven dependency stanza I'm using. Please
> >> let me know if I've got it wrong.
> >>
> >> Thanks,
> >> Nick
> >>
> >>     <dependency>
> >>       <groupId>org.apache.flink</groupId>
> >>       <artifactId>flink-streaming-core</artifactId>
> >>       <version>${flink.version}</version>
> >>       <classifier>test</classifier>
> >>       <scope>test</scope>
> >>     </dependency>
> >
> >
>

Mime
View raw message