flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Iterative stream processing - cyclic JobGraph?
Date Tue, 08 Jul 2014 14:16:14 GMT
For extending the feature set of the streaming project today we were
experimenting with the addition of an IterativeDataStream class, resulting
in the following dummy application:

https://github.com/stratosphere/stratosphere-streaming/blob/iterate/stratosphere-streaming-core/src/test/java/eu/stratosphere/streaming/api/IterateTest.java#L80-89

This approach constructs a cyclic JobGraph and exception is thrown for it
at the JobManager:

https://github.com/apache/incubator-flink/blob/master/stratosphere-runtime/src/main/java/eu/stratosphere/nephele/jobmanager/JobManager.java#L419-424

However when removing this check the JobGraph was created and working as
expected.

Is the check really needed or should we take another approach when
constructing the iterative jobs?

Thanks,

Marton

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message