incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sky Zhao <>
Subject RE: zeroMQ
Date Wed, 08 May 2013 02:28:05 GMT
Hi Kishore G,

Using batching events, need to set interval time in data sender side, in fact, so far I have
no use case to further discuss, just think the solution once having big data scenario,e.g.
mobile/internet traffic data,

Thanks a lot!


From: kishore g []
Sent: Tuesday, May 07, 2013 11:40 PM
Subject: Re: zeroMQ

Hi Sky,

To increase throughput, you might need batching of events to save on the round trip costs.
But I am curious as to why you need that. Would you mind providing some additional details
about your use case and its primary goals.

Kishore G

On Tue, May 7, 2013 at 2:28 AM, Sky Zhao <<>>
Thanks Matthieu,

If do the blocking, event arriving rate must be lower than processing rate, or loss event,
so need to config the parameters in file(s4.remoteSender.maxRate,s4.sender.maxRate?).
This is the way of  limiting the throughput, but if the data throughput is very large, is
there any other way to do the blocking events?


From: Matthieu Morel [<>]
Sent: Tuesday, May 07, 2013 4:46 PM

Subject: Re: zeroMQ

On May 7, 2013, at 03:35 , Sky Zhao wrote:

Thanks Matthieu,

how implemet the blocking senders and receivers? can you give me examples or links?

Have a look here:



From: Matthieu Morel [<>]
Sent: Monday, May 06, 2013 6:06 PM
Subject: Re: zeroMQ

It is already possible to prevent losing events in normal operation in S4 by using blocking
senders and receivers. See the 0.6 documentation.

Even though zeromq has some interesting features such as automatic reconnection, providing
"no loss" guarantee in the case of node failures would require application level acknowledgement.
In other words, transport layer implementation does not really matter, in my opinion. We have
envisaged some designs but there is no plan for implementing application-level ack at the

Hope this helps,


On May 6, 2013, at 04:44 , Sky Zhao wrote:

Can s4 plan to use zeroMQ to handle events to improve current event loss issue in future?


View raw message