flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saiph Kappa <saiph.ka...@gmail.com>
Subject Re: Flink Stream: collect in an array all records within a window
Date Mon, 18 Jan 2016 22:21:15 GMT
Hi Matthias,

Thanks for your response. The method .writeToSocket seems to be what I was
looking for. Can you tell me what kind of serialization schema should I use
assuming my socket server receives strings. I have something like this in
scala:

val server = new ServerSocket(9999)while (true) {
    val s = server.accept()
    val in = new BufferedSource(s.getInputStream()).getLines()
    println(in.next())
    s.close()}

Thanks



On Mon, Jan 18, 2016 at 8:46 PM, Matthias J. Sax <mjsax@apache.org> wrote:

> Hi Saiph,
>
> you can use AllWindowFunction via .apply(...) to get an .collect method:
>
> From:
>
> https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/streaming_guide.html
>
> > // applying an AllWindowFunction on non-keyed window stream
> > allWindowedStream.apply (new AllWindowFunction<Tuple2<String,Integer>,
> Integer, Window>() {
> >     public void apply (Window window,
> >             Iterable<Tuple2<String, Integer>> values,
> >             Collector<Integer> out) throws Exception {
> >         int sum = 0;
> >         for (value t: values) {
> >             sum += t.f1;
> >         }
> >         out.collect (new Integer(sum));
> >     }
> > });
>
> If you consume all those value via an sink, the sink will run an the
> cluster. You can use .writeToSocket(...) as sink:
>
> https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/streaming_guide.html#data-sinks
>
> -Matthias
>
>
> On 01/18/2016 06:30 PM, Saiph Kappa wrote:
> > Hi,
> >
> > After performing a windowAll() on a DataStream[String], is there any
> > method to collect and return an array with all Strings within a window
> > (similar to .collect in Spark).
> >
> > I basically want to ship all strings in a window to a remote server
> > through a socket, and want to use the same socket connection for all
> > strings that I send. The method .addSink iterates over all records, but
> > does the provided function runs on the flink client or on the server?
> >
> > Thanks.
>
>

Mime
View raw message