flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2727) Add a base class for MessageQueue-with-acknowledgement sources
Date Thu, 01 Oct 2015 16:19:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14940030#comment-14940030
] 

ASF GitHub Bot commented on FLINK-2727:
---------------------------------------

Github user mxm commented on the pull request:

    https://github.com/apache/flink/pull/1163#issuecomment-144777018
  
    Valuable base class for integrating messaging queuing systems. Looks good.


> Add a base class for MessageQueue-with-acknowledgement sources
> --------------------------------------------------------------
>
>                 Key: FLINK-2727
>                 URL: https://issues.apache.org/jira/browse/FLINK-2727
>             Project: Flink
>          Issue Type: New Feature
>          Components: Streaming
>    Affects Versions: 0.10
>            Reporter: Stephan Ewen
>            Assignee: Stephan Ewen
>
> Several message queues (RabbitMQ, Amazon SQS) have the pattern that you retrieve messages
and acknowledge them back by ID.
> We can create a simple base non-parallel source that provides tooling for:
>   - Collecting the IDs of elements emitted between two checkpoints
>   - Persisting them with the checkpoint, respecting proper serialization
>   - Acknowledging them when a checkpoint is notified of completion.
> This assumes that the Message Queues retain unacknowledged messages and re-emit them
after the acknowledgement period expired. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message