flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Holzemer (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1846) Sinks inside of iterations
Date Wed, 08 Apr 2015 12:43:13 GMT
Markus Holzemer created FLINK-1846:
--------------------------------------

             Summary: Sinks inside of iterations
                 Key: FLINK-1846
                 URL: https://issues.apache.org/jira/browse/FLINK-1846
             Project: Flink
          Issue Type: Improvement
          Components: Distributed Runtime, Local Runtime, Optimizer
            Reporter: Markus Holzemer
            Assignee: Markus Holzemer
            Priority: Minor


I startet work on adding support for file sinks inside of iterations to flink.
My current plan includes two different modes: one where only the output file of the last iteration
is kept and one where the output file of every single iteration is kept (the corresponding
superstep number is added as a prefix).

I am also thinking about adding a mode where the output of every iteration is appended to
one file or where a "sliding window" of n files is kept for all iterations.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message