flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Hogan <c...@greghogan.com>
Subject Iteration Intermediate Output
Date Thu, 26 May 2016 20:41:53 GMT
Hi y'all,

I think this is an oft-requested feature [0] and there are many graph
algorithms for which intermediate output is the desired result. I'd like to
take Stephan up on his offer [1] for pointers.

I have yet to get in deep, but I see that iteration tasks are treated
specially as IterationIntermediateTask for synchronization between
supersteps. Also, when OperatorTranslation and GraphCreatingVisitor are
walking the program DAG an iteration must be first reached through the tail.

Greg

[0]
http://stackoverflow.com/questions/37224140/possibility-of-saving-partial-outputs-from-bulk-iteration-in-flink-dataset
[1]
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Intermediate-output-during-delta-iterations-td436.html

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message