flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abdul Salam Shaikh <abd.salam.sha...@gmail.com>
Subject Applying the same operator twice on a windowed stream
Date Fri, 27 Jan 2017 20:27:16 GMT
Hi everyone,

I have a window definition like this at the moment in snapshot version
1.2.0:

 final StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
        env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
        env.setParallelism(1);
        DataStream<String> live = env.addSource(new JsonTestSource());
        DataStream<FlatObject> jsonToTuple = live.flatMap(new
RawEventTransformer());

        KeyedStream<FlatObject, String> keyStream =  jsonToTuple.keyBy(new
KeySelector<FlatObject,String>() {
        public String getKey(FlatObject value) throws Exception {
            return value.getIntersectionName();
           }
        });

        DataStream<FlatObject> flatCorrectedStream =
                            keyStream
                            .window(GlobalWindows.create())
                            .trigger(new WindowCustomTrigger())
                            .evictor(new WindowEvictor())
                            .apply(new TrafficWindow());
        flatCorrectedStream.print();

My apply function generally corrects the raw event streams.

I want to evaluate the corrected stream and generate a stream of
EvaluatedWindowObjectsStream.

However,  using the same operator twice is not an allowed:

 DataStream<FlatObject> flatCorrectedStream =
                            keyStream
                            .window(GlobalWindows.create())
                            .trigger(new WindowCustomTrigger())
                            .evictor(new WindowEvictor())
                            .apply(new TrafficWindow());

*                                   .apply(new WindowEvaluater());*

I am looking for options where I can achieve the above case. I am looking
to keep the logic of correcting the streams and evaluating the streams
separately. Hence, the above case rises.

​Thanks!​

Mime
View raw message