incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karthik Kambatla <kkamb...@cs.purdue.edu>
Subject Re: Thoughts on adding guaranteed message processing
Date Mon, 06 Aug 2012 20:11:40 GMT
Hi Benjamin,

That is right. At the moment, S4 does not implement point 3. Even if it
doesn't exist at the moment, it is definitely one of the important items on
our roadmap.

Status:

   1. We have made attempts at addressing this in
   https://issues.apache.org/jira/browse/S4-7
   2. Also, there is another JIRA open for this purpose -
   https://issues.apache.org/jira/browse/S4-41

Please feel free to re-assign it to yourself, should you decide to
implement it.

Flavio - it is indeed tricky to offer exactly-once semantics. My
understanding is that the underlying comm-layer could filter out subsequent
duplicate events; however, we need to sacrifice ordering.

Thanks
Karthik

On Mon, Aug 6, 2012 at 6:43 AM, "Benjamin Süß" <Gothic13@gmx.de> wrote:

> Hi Matthieu,
>
> thank you for your reply. I had a specific use case in mind, indeed:
>
> I am trying to track RFID tags in distributed systems. This means, that
> not even a single tag scan may get lost. And of course, none are to be sent
> twice or even more often as this would heavily confuse any surveillance
> routines I am going to implement.
>
> Regarding your answers, especially point 3, I do not think this can be
> done with S4 at the moment, can it?
>
> Regards,
> Benjamin
>
> -------- Original-Nachricht --------
> > Datum: Tue, 31 Jul 2012 17:30:27 +0200
> > Von: Matthieu Morel <mmorel@apache.org>
> > An: s4-user@incubator.apache.org
> > Betreff: Re: Thoughts on adding guaranteed message processing
>
> > On 7/31/12 2:54 PM, "Benjamin Süß" wrote:
> > > Hi there,
> > >
> > > it is stated in several places that S4 does not include guaranteed
> > one-time message processing. So my question is: are there currently any
> plans on
> > adding this to S4? Or is it certain this is not going to happen? If there
> > are any plans on this, can I find further information somewhere?
> > >
> >
> > There are typically 3 requirements for guaranteeing one-time message
> > processing:
> >
> > 1. reliable communication channels
> >
> > 2. replayable input stream: you need an upstream component that is able
> > to store/bufferize the whole stream and replay on demand.
> >
> > 3. tracking of messages, using some sort of piggybacking, possibly
> > requiring manual input from the user.
> >
> >
> > In S4 0.5.0, we already address 1. by providing communications through
> > TCP by default. Requirement 2. is quite straightforward to implement, by
> > adding some machinery to connect to a component such as Apache Kafka for
> > instance. We are considering options for 3.
> >
> >
> > Do you have a specific use case in mind?
> >
> > Regards,
> >
> > Matthieu
> >
>

Mime
View raw message