Return-Path: X-Original-To: apmail-flink-dev-archive@www.apache.org Delivered-To: apmail-flink-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2E55D17E4C for ; Sun, 4 Jan 2015 11:00:55 +0000 (UTC) Received: (qmail 196 invoked by uid 500); 4 Jan 2015 11:00:56 -0000 Delivered-To: apmail-flink-dev-archive@flink.apache.org Received: (qmail 135 invoked by uid 500); 4 Jan 2015 11:00:56 -0000 Mailing-List: contact dev-help@flink.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.incubator.apache.org Delivered-To: mailing list dev@flink.incubator.apache.org Received: (qmail 124 invoked by uid 99); 4 Jan 2015 11:00:55 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 04 Jan 2015 11:00:55 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Sun, 04 Jan 2015 11:00:55 +0000 Received: (qmail 99800 invoked by uid 99); 4 Jan 2015 11:00:34 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 04 Jan 2015 11:00:34 +0000 Date: Sun, 4 Jan 2015 11:00:34 +0000 (UTC) From: "Paris Carbone (JIRA)" To: dev@flink.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (FLINK-1344) Add support for lazy collections and scala streams for functional source definitions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org Paris Carbone created FLINK-1344: ------------------------------------ Summary: Add support for lazy collections and scala streams for functional source definitions Key: FLINK-1344 URL: https://issues.apache.org/jira/browse/FLINK-1344 Project: Flink Issue Type: New Feature Components: Streaming Reporter: Paris Carbone Assignee: Paris Carbone Priority: Trivial Source definitions in the scala-api pass a collector to the UDF, thus enforcing an imperative style for defining custom streams. In order maintain a purely functional coding style in the streaming scala-api while also adding some interoperability with scala constructs it would be nice to add a source definition that gets a generating lazily collection or a scala/akka stream and just delegates internally the calls to the collector while iterating them. Mind that source parallelism should be 1. As an additional idea, sinks could also support wrapping up flink streams to scala streams for full interoperability with scala streaming code. -- This message was sent by Atlassian JIRA (v6.3.4#6332)