curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Zimmerman <jor...@jordanzimmerman.com>
Subject Re: Using lockPath() in QueueBuilder can hurt the throughput by 134%
Date Thu, 28 Nov 2013 17:30:36 GMT
This doesn’t surprise me. The Javadoc for the lockPath() method even says "There is a small
performance penalty for this behavior however.”. I guess we should update that, but taking
a lock for every message will incur a performance penalty.

-Jordan

On Nov 28, 2013, at 4:48 AM, Sznajder ForMailingList <bs4mailinglist@gmail.com> wrote:

> Hi
> 
> Following my precedent questions on this mailing list, I continued to investigate and
found that using lockPath() in the QueueBuilder can hurt the throughput of the consumption
of the queue items by 134%.
> 
> In fact the lockPath() starts to seriously hurting the throughput when we multiply the
number of queue consumers.
> 
> 
> For helping you to reproduce the problem, I wrote the attached code.
> 
> The main class gets one parameter : the connectString to the ZQ quorum. If no parameters
is provided, the class will launch the Curator TestingServer.
> 
> 
> The producer starts by putting 100 items in the queue, then it monitors the number of
elements in the queue and measures the time spent to consume the whole queue.
> 
> The consumers just consume the items without doing anything else, and then sleep 500
msecs.
> 
> I tested the program with these different parameters:
> 1, 2, 4, 8, 16 and 32 consumers...
> with setting the lockPath() and without setting the lockPath().
> 
> 
> 
> And I get the number described in the attached Excel file. You can see that for 1 or
2 consumers, the hurt by using lockPath() isn't so significant, but it can reach 134% when
we use a larger number of consumers....
> 
> 
> I put here the output:
> 
>      [java] Without LOCK_PATH
>      [java] -=-=-=-=-=-= 1 consumers =-=-=-=-=-=-
>      [java] It took 29562 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 32 consumers =-=-=-=-=-=-
>      [java] It took 1559 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 16 consumers =-=-=-=-=-=-
>      [java] It took 2022 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 8 consumers =-=-=-=-=-=-
>      [java] It took 4045 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 4 consumers =-=-=-=-=-=-
>      [java] It took 7541 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 2 consumers =-=-=-=-=-=-
>      [java] It took 15539 msecs for consuming 100 items
>      [java]
>      [java]
>      [java]
>      [java] With LOCK_PATH
>      [java] -=-=-=-=-=-= 1 consumers =-=-=-=-=-=-
>      [java] It took 30562 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 32 consumers =-=-=-=-=-=-
>      [java] It took 3639 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 16 consumers =-=-=-=-=-=-
>      [java] It took 4610 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 8 consumers =-=-=-=-=-=-
>      [java] It took 6101 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 4 consumers =-=-=-=-=-=-
>      [java] It took 9073 msecs for consuming 100 items
>      [java] -=-=-=-=-=-= 2 consumers =-=-=-=-=-=-
>      [java] It took 16053 msecs for consuming 100 items
> 
> <QueueConsumerProducer.java><SimpleQueueItem.java><Throughput.pdf>


Mime
View raw message