incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: throttling vs blocking
Date Wed, 17 Jul 2013 18:40:25 GMT

On Jul 17, 2013, at 19:58 , Jiaan Zeng <l.allen09@gmail.com> wrote:

> On Wed, Jul 17, 2013 at 12:12 PM, Matthieu Morel <mmorel@apache.org> wrote:
>> 
>> On Jul 17, 2013, at 17:16 , Jiaan Zeng <l.allen09@gmail.com> wrote:
>> 
>>> Hi,
>>> 
>>> 1)
>>> I wonder what is the main difference between
>>> org.apache.s4.comm.staging.ThrottlingThreadPoolExecutorService and
>>> org.apache.s4.comm.staging.BlockingThreadPoolExecutorService
>>> 
>>> I understand ThrottlingThreadPoolExecutorService uses RateLimiter to
>>> control the rate while BlockingThreadPoolExecutorService uses
>>> Semaphore. I guess ThrottlingThreadPoolExecutorService is supposed to
>>> drop events while BlockingThreadPoolExecutorService is supposed to
>>> block until thread pool can handle events.
>>> 
>>> However, in ThrottlingThreadPoolExecutorService -> submit(Runnable
>>> task), rateLimitedPermits.acquire() is called which means the
>>> invocation is blocked until permit is acquired. It seems to me that
>>> ThrottlingThreadPoolExecutorService never drops events and works in
>>> the same way as BlockingThreadPoolExecutorService does. And there is
>>> no unit test for event drop on this.
>> 
>> For throttling, trying to acquire faster than the max rate will fill up the work
queue of the thread pool and new tasks will be dropped (rejected by the executor).
>> For blocking, we control the work queue directly through semaphores, and that's blocking.
>> 
> 
> The throttling does not seem to drop event. That is my question.
> Because it calls rateLimitedPermits.acquire() instead of tryAcquire(),
> which blocks until thread pool can handle.

but in the meantime tasks enqueue in the work queue of the threadpool, and when that work
queue is full, tasks are rejected, therefore dropped.

Have another look at the code and the difference between waiting for _executing a task_ (
as in  "throttling" ) and waiting for _enqueuing a task_ (as in "blocking" in our case).

> 
> Are there any test cases that I can try on this?

just swap implementations by defining your custom module

Matthieu




> 
> 
> 
> --
> Regards,
> Jiaan


Mime
View raw message