flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From defstat <defs...@gmail.com>
Subject FLink Streaming - Parallelize ArrayList
Date Tue, 29 Sep 2015 02:29:37 GMT
Hi all,

Is there a way to either Broadcast or Parallelize  an ArrayList of custom
objects? 

The application is as follows:

I have 2 streams taken from two sockets. The first stream contains vectors
that should populate one list of vectors by deciding whether an arriving
vector is part of the list, after calculations made on the existing list
vectors and the new one.

The other stream is also containing vectors that use the above list, and
sorts it using the info contained in the incoming vectors.

I tried to solve this using  

private static final ArrayList<Vector> aSetOfVectors = new
ArrayList<Vector>();

vectorsFirstStream.fold(aSetOfVectors , new FoldFunction<Vector,
ArrayList&lt;Vector>>() {
 .... (Calculations and new aSetOfVectors produced) ......

return aSetOfVectors;
}

vectorsSecondStream.fold(aSetOfVectors , new FoldFunction<Vector,
ArrayList&lt;Vector>>() {
 .... (Calculations and sorted vector list produced) ......
}

I think that that approach could work only locally. Is there a way to
parallelize the above?

Thank you in advance




--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/FLink-Streaming-Parallelize-ArrayList-tp2957.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message