flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-2489) Improve the streaming window join
Date Wed, 05 Aug 2015 20:00:05 GMT
Stephan Ewen created FLINK-2489:

             Summary: Improve the streaming window join
                 Key: FLINK-2489
                 URL: https://issues.apache.org/jira/browse/FLINK-2489
             Project: Flink
          Issue Type: Improvement
          Components: Streaming
    Affects Versions: 0.10
            Reporter: Stephan Ewen

This issue tracks the ideas and discussion to improve the current window join.

Here is an initial list of desiderata for an improved window join:

  - Joins should initially run only on time windows
  - Support for operator time and event time
  - Event time version must support multiple windows being in progress (or buffered) while
waiting for watermarks

  - Support for simple Java heap and Flink-managed memory
  - Flink-managed memory variant should support out-of-core operation

  - Join buffers or hashtables need to be checkpointed.
  - Join buffers or hashtables need to support incremental checkpointing
  - Join buffers or hashtables should support asynchronous checkpointing

  - For sliding time windows, we should consider variants to reuse the join candidates across
multiple windows they are contained in.

This message was sent by Atlassian JIRA

View raw message