incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <>
Subject Re: about the benchmark injector and hotspot
Date Wed, 05 Jun 2013 08:18:10 GMT

On Jun 4, 2013, at 16:00 , baojian Zhou wrote:

> thank you Mr Matthieu,
> if i want to configure the load shedding, do i just need to add the
> parameters such as s4.sender.workQueueSize
> s4.sender.maxRate in the file?

These parameters are for a throttling remote sender (which might also do load shedding). You'd
need to define an overriding module for replacing the binding for RemoteSendersExecutorServiceFactory
with a factory for a throttling executor to use it. See here for customization:

But also note you can perform load shedding in the consumer.

I'd suggest to have a look here:

and also to have a look at the code, that's the best way to clearly understand the framework.


> 2013/6/4 Matthieu Morel <>
>> On Jun 4, 2013, at 13:09 , baojian Zhou wrote:
>>> hi all
>>> i want to do an experiment in the benchmark package and test the
>>> injector-rate and received-events, but i get the following confuses:
>>> 1. i test two times in the same parameters, but in each time, the number
>> of
>>> input events in the injector are different:
>>> the first: 3324831
>>> the second: 9808773
>>> and the two experiments have the same parameters:
>>> -c=testCluster2 -appName=adapter -zk=node4:2181
>>> i want to expect that very time the injectors have the same rate and
>> finish
>>> in definite number of events. can i complete the work and how ?
>> Note that the rate is not deterministic.
>> Are you sure you are correctly reading the number of injected events? If
>> you lose events, that could be due to load shedding (if you configured the
>> platform to use that) or injecting before the all consumers are ready.
>>> 2. and i do not very clear about the dequeued csv files, dose it stand
>> for
>>> the number of the dequeued streams in given time interval. if it is true,
>>> how can i get the the number of the events in the streams in very given
>>> time interval ?
>> The output provides total dequeued events, average over the whole run, and
>> 1, 5, 15 minutes averages. Please have a look at the metrics documentation
>> for more info.
>> Matthieu

View raw message