Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 0BBB0200B74 for ; Thu, 1 Sep 2016 18:24:23 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 0A3F7160AB5; Thu, 1 Sep 2016 16:24:23 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 78474160AA8 for ; Thu, 1 Sep 2016 18:24:22 +0200 (CEST) Received: (qmail 12102 invoked by uid 500); 1 Sep 2016 16:24:20 -0000 Mailing-List: contact issues-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list issues@flink.apache.org Received: (qmail 12049 invoked by uid 99); 1 Sep 2016 16:24:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 01 Sep 2016 16:24:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id A9B7C2C1B77 for ; Thu, 1 Sep 2016 16:24:20 +0000 (UTC) Date: Thu, 1 Sep 2016 16:24:20 +0000 (UTC) From: "Elias Levy (JIRA)" To: issues@flink.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (FLINK-4558) Add support for synchronizing streams MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 01 Sep 2016 16:24:23 -0000 Elias Levy created FLINK-4558: --------------------------------- Summary: Add support for synchronizing streams Key: FLINK-4558 URL: https://issues.apache.org/jira/browse/FLINK-4558 Project: Flink Issue Type: Improvement Components: Streaming Affects Versions: 1.1.0 Reporter: Elias Levy As mentioned on the [mailing list|http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/synchronizing-two-streams-td6830.html], there are use cases that require synchronizing two streams on via their times and where it is not practical to buffer all messages from one streams while waiting for the other to synchronize. Flink should add functionality to enable such use cases. This could be implemented by modifying TwoInputStreamOperator so that calls to processElement1 and processElement2 could return a value indicating that the element can't yet be processed, having the framework then pause processing for some time, potentially using exponential back off with a hard maximum, and then allowing the back pressure system to do its work and pause the stream. Alternatively, an API could be added to explicitly pause/unpause a stream. For ease of use either of these mechanism should be used to create a SynchronizedTwoInputStreamOperator that end users can utilize by passing a configurable time delta to use as a synchronization threshold. -- This message was sent by Atlassian JIRA (v6.3.4#6332)