flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paris Carbone (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1344) Add support for lazy collections and scala streams for functional source definitions
Date Sun, 04 Jan 2015 11:00:34 GMT
Paris Carbone created FLINK-1344:
------------------------------------

             Summary: Add support for lazy collections and scala streams for functional source
definitions
                 Key: FLINK-1344
                 URL: https://issues.apache.org/jira/browse/FLINK-1344
             Project: Flink
          Issue Type: New Feature
          Components: Streaming
            Reporter: Paris Carbone
            Assignee: Paris Carbone
            Priority: Trivial


Source definitions in the scala-api pass a collector to the UDF, thus enforcing an imperative
style for defining custom streams. In order maintain a purely functional coding style in the
streaming scala-api while also adding some interoperability with scala constructs it would
be nice to add a source definition that gets a generating lazily collection or a scala/akka
stream and just delegates internally the calls to the collector while iterating them. Mind
that source parallelism should be 1.

As an additional idea, sinks could also support wrapping up flink streams to scala streams
for full interoperability with scala streaming code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message