incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Junqueira <...@yahoo-inc.com>
Subject Re: Thoughts on adding guaranteed message processing
Date Tue, 31 Jul 2012 15:35:38 GMT
Interesting question. Does one-time imply exactly-once semantics? Out of curiosity, how would
you define exactly-once semantics for a stream processing platform?

-Flavio

On Jul 31, 2012, at 5:30 PM, Matthieu Morel wrote:

> On 7/31/12 2:54 PM, "Benjamin Süß" wrote:
>> Hi there,
>> 
>> it is stated in several places that S4 does not include guaranteed one-time message
processing. So my question is: are there currently any plans on adding this to S4? Or is it
certain this is not going to happen? If there are any plans on this, can I find further information
somewhere?
>> 
> 
> There are typically 3 requirements for guaranteeing one-time message 
> processing:
> 
> 1. reliable communication channels
> 
> 2. replayable input stream: you need an upstream component that is able 
> to store/bufferize the whole stream and replay on demand.
> 
> 3. tracking of messages, using some sort of piggybacking, possibly 
> requiring manual input from the user.
> 
> 
> In S4 0.5.0, we already address 1. by providing communications through 
> TCP by default. Requirement 2. is quite straightforward to implement, by 
> adding some machinery to connect to a component such as Apache Kafka for 
> instance. We are considering options for 3.
> 
> 
> Do you have a specific use case in mind?
> 
> Regards,
> 
> Matthieu
> 


Mime
View raw message