flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@apache.org>
Subject Re: Flink Stream: collect in an array all records within a window
Date Tue, 19 Jan 2016 09:34:46 GMT
There is SimpleStringSchema.

-Matthias

On 01/18/2016 11:21 PM, Saiph Kappa wrote:
> Hi Matthias,
> 
> Thanks for your response. The method .writeToSocket seems to be what I
> was looking for. Can you tell me what kind of serialization schema
> should I use assuming my socket server receives strings. I have
> something like this in scala:
> 
> |val server =newServerSocket(9999)while(true){val s =server.accept()val
> in=newBufferedSource(s.getInputStream()).getLines()println(in.next())s.close()}
> 
> |
> 
> Thanks|
> |
> 
>  
> 
> On Mon, Jan 18, 2016 at 8:46 PM, Matthias J. Sax <mjsax@apache.org
> <mailto:mjsax@apache.org>> wrote:
> 
>     Hi Saiph,
> 
>     you can use AllWindowFunction via .apply(...) to get an .collect method:
> 
>     From:
>     https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/streaming_guide.html
> 
>     > // applying an AllWindowFunction on non-keyed window stream
>     > allWindowedStream.apply (new
>     AllWindowFunction<Tuple2<String,Integer>, Integer, Window>() {
>     >     public void apply (Window window,
>     >             Iterable<Tuple2<String, Integer>> values,
>     >             Collector<Integer> out) throws Exception {
>     >         int sum = 0;
>     >         for (value t: values) {
>     >             sum += t.f1;
>     >         }
>     >         out.collect (new Integer(sum));
>     >     }
>     > });
> 
>     If you consume all those value via an sink, the sink will run an the
>     cluster. You can use .writeToSocket(...) as sink:
>     https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/streaming_guide.html#data-sinks
> 
>     -Matthias
> 
> 
>     On 01/18/2016 06:30 PM, Saiph Kappa wrote:
>     > Hi,
>     >
>     > After performing a windowAll() on a DataStream[String], is there any
>     > method to collect and return an array with all Strings within a window
>     > (similar to .collect in Spark).
>     >
>     > I basically want to ship all strings in a window to a remote server
>     > through a socket, and want to use the same socket connection for all
>     > strings that I send. The method .addSink iterates over all
>     records, but
>     > does the provided function runs on the flink client or on the server?
>     >
>     > Thanks.
> 
> 


Mime
View raw message