incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiaan Zeng <l.alle...@gmail.com>
Subject Re: throttling vs blocking
Date Wed, 17 Jul 2013 21:42:16 GMT
That makes sense. Thank.

On Wed, Jul 17, 2013 at 2:40 PM, Matthieu Morel <mmorel@apache.org> wrote:
>
> On Jul 17, 2013, at 19:58 , Jiaan Zeng <l.allen09@gmail.com> wrote:
>
>> On Wed, Jul 17, 2013 at 12:12 PM, Matthieu Morel <mmorel@apache.org> wrote:
>>>
>>> On Jul 17, 2013, at 17:16 , Jiaan Zeng <l.allen09@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> 1)
>>>> I wonder what is the main difference between
>>>> org.apache.s4.comm.staging.ThrottlingThreadPoolExecutorService and
>>>> org.apache.s4.comm.staging.BlockingThreadPoolExecutorService
>>>>
>>>> I understand ThrottlingThreadPoolExecutorService uses RateLimiter to
>>>> control the rate while BlockingThreadPoolExecutorService uses
>>>> Semaphore. I guess ThrottlingThreadPoolExecutorService is supposed to
>>>> drop events while BlockingThreadPoolExecutorService is supposed to
>>>> block until thread pool can handle events.
>>>>
>>>> However, in ThrottlingThreadPoolExecutorService -> submit(Runnable
>>>> task), rateLimitedPermits.acquire() is called which means the
>>>> invocation is blocked until permit is acquired. It seems to me that
>>>> ThrottlingThreadPoolExecutorService never drops events and works in
>>>> the same way as BlockingThreadPoolExecutorService does. And there is
>>>> no unit test for event drop on this.
>>>
>>> For throttling, trying to acquire faster than the max rate will fill up the work
queue of the thread pool and new tasks will be dropped (rejected by the executor).
>>> For blocking, we control the work queue directly through semaphores, and that's
blocking.
>>>
>>
>> The throttling does not seem to drop event. That is my question.
>> Because it calls rateLimitedPermits.acquire() instead of tryAcquire(),
>> which blocks until thread pool can handle.
>
> but in the meantime tasks enqueue in the work queue of the threadpool, and when that
work queue is full, tasks are rejected, therefore dropped.
>
> Have another look at the code and the difference between waiting for _executing a task_
( as in  "throttling" ) and waiting for _enqueuing a task_ (as in "blocking" in our case).
>
>>
>> Are there any test cases that I can try on this?
>
> just swap implementations by defining your custom module
>
> Matthieu
>
>
>
>
>>
>>
>>
>> --
>> Regards,
>> Jiaan
>



-- 
Regards,
Jiaan

Mime
View raw message