curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sznajder ForMailingList <bs4mailingl...@gmail.com>
Subject Using lockPath() in QueueBuilder can hurt the throughput by 134%
Date Thu, 28 Nov 2013 12:48:57 GMT
Hi

Following my precedent questions on this mailing list, I continued to
investigate and found that using lockPath() in the QueueBuilder can hurt
the throughput of the consumption of the queue items by 134%.

In fact the lockPath() starts to seriously hurting the throughput when we
multiply the number of queue consumers.


For helping you to reproduce the problem, I wrote the attached code.

The main class gets one parameter : the connectString to the ZQ quorum. If
no parameters is provided, the class will launch the Curator TestingServer.


The producer starts by putting 100 items in the queue, then it monitors the
number of elements in the queue and measures the time spent to consume the
whole queue.

The consumers just consume the items without doing anything else, and then
sleep 500 msecs.

I tested the program with these different parameters:
1, 2, 4, 8, 16 and 32 consumers...
with setting the lockPath() and without setting the lockPath().



And I get the number described in the attached Excel file. You can see that
for 1 or 2 consumers, the hurt by using lockPath() isn't so significant,
but it can reach 134% when we use a larger number of consumers....


I put here the output:

     [java] Without LOCK_PATH
     [java] -=-=-=-=-=-= 1 consumers =-=-=-=-=-=-
     [java] It took 29562 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 32 consumers =-=-=-=-=-=-
     [java] It took 1559 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 16 consumers =-=-=-=-=-=-
     [java] It took 2022 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 8 consumers =-=-=-=-=-=-
     [java] It took 4045 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 4 consumers =-=-=-=-=-=-
     [java] It took 7541 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 2 consumers =-=-=-=-=-=-
     [java] It took 15539 msecs for consuming 100 items
     [java]
     [java]
     [java]
     [java] With LOCK_PATH
     [java] -=-=-=-=-=-= 1 consumers =-=-=-=-=-=-
     [java] It took 30562 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 32 consumers =-=-=-=-=-=-
     [java] It took 3639 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 16 consumers =-=-=-=-=-=-
     [java] It took 4610 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 8 consumers =-=-=-=-=-=-
     [java] It took 6101 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 4 consumers =-=-=-=-=-=-
     [java] It took 9073 msecs for consuming 100 items
     [java] -=-=-=-=-=-= 2 consumers =-=-=-=-=-=-
     [java] It took 16053 msecs for consuming 100 items

Mime
View raw message