flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Kruse (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1038) Adding a collection output format
Date Tue, 29 Jul 2014 12:47:38 GMT
Sebastian Kruse created FLINK-1038:

             Summary: Adding a collection output format
                 Key: FLINK-1038
                 URL: https://issues.apache.org/jira/browse/FLINK-1038
             Project: Flink
          Issue Type: Improvement
            Reporter: Sebastian Kruse
            Priority: Minor

Similar to the existing LocalCollectionOutputFormat or Spark's collect() method, it would
be nice to have a CollectionOutputFormat that also works when running jobs on a cluster. This
output format gathers all results of a sink from all TaskManagers in the JVM that submitted
the job plan and provides these as a collection, similar to accumulators. After all, this
can help to avoid the tedious task of going to HDFS and read and parse the single result files.

PS. We have already created such an output format and can contribute it.

This message was sent by Atlassian JIRA

View raw message