qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Lavigne du Cadet" <j.lavigneduca...@abc-arbitrage.com>
Subject RE: Qpid C++ broker memory leak?
Date Fri, 24 Sep 2010 07:40:37 GMT
Here's the output I have for a few run. The memory consumption pattern seems quite different
from yours but it doesn't increase linearly

abcqpid@abcQpid:~/qpid-0.5/cpp/src/tests$ while ps up `pgrep qpidd`; do ./perftest -s --username
guest --password guest123 --size 64 --count 1000000 ; done
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  0.0  0.0 244104  6020 ?        Ssl  09:08   0:00 qpidd -d
123551  111219  223287  13.6284
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167 31.2  2.1 487524 174900 ?       Ssl  09:08   0:16 qpidd -d
122093  101291  203355  12.4118
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167 53.6  3.5 562036 287208 ?       Ssl  09:08   0:33 qpidd -d
127229  111748  223542  13.6439
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167 69.5  3.4 562036 285792 ?       Ssl  09:08   0:50 qpidd -d
128199  105704  212375  12.9623
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167 82.4  5.3 758748 441024 ?       Rsl  09:08   1:06 qpidd -d
123956  109960  219983  13.4267
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167 92.7  4.1 627676 340588 ?       Ssl  09:08   1:23 qpidd -d
129984  112129  224308  13.6907
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  100  5.0 693212 414732 ?       Ssl  09:08   1:39 qpidd -d
107284  106511  213869  13.0535
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  106  4.1 693212 340740 ?       Ssl  09:08   1:56 qpidd -d
101739  100153  200296  12.2251
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  112  4.3 693212 354076 ?       Ssl  09:08   2:13 qpidd -d
128088  108798  217643  13.2839
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  117  5.5 776172 455168 ?       Ssl  09:08   2:30 qpidd -d
129048  112187  225225  13.7466
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  121  4.7 758748 391288 ?       Ssl  09:08   2:46 qpidd -d
128988  112805  225689  13.7749
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  124  5.2 758880 430636 ?       Ssl  09:08   3:02 qpidd -d
131103  112403  225844  13.7845
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  128  5.9 824416 487504 ?       Ssl  09:08   3:18 qpidd -d
129258  113177  226364  13.8162
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  130  4.9 758880 403288 ?       Ssl  09:08   3:34 qpidd -d
120588  109785  219620  13.4046
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     20167  133  5.3 758880 439848 ?       Ssl  09:08   3:50 qpidd -d


-----Message d'origine-----
De : Alan Conway [mailto:aconway@redhat.com] 
Envoyé : jeudi 23 septembre 2010 19:20
À : users@qpid.apache.org
Cc : Julien Lavigne du Cadet
Objet : Re: Qpid C++ broker memory leak?

On 09/23/2010 08:43 AM, Julien Lavigne du Cadet wrote:
> Hi,
>
>
>
> I noticed something that looks like a memory leak with both qpid 0.5 and
> 0.6 on ubuntu (hardy) and I would like your feedback.
>
>
>
> I have a test application that publishes small non persistent messages
> as fast as possible.
>
> Another application connects to the broker and creates a queue with auto
> delete set to true. I don't read the message in order to fill the queue
> with hundreds of thousands of messages.  After a few seconds, the broker
> consumes 2GB of memory and I kill both clients. I can see in qmf that
> the queue is deleted correctly, however qpid will release only a very
> small portion of the memory (for 2GB, it will be around 200MB). Waiting
> doesn't change a thing. If I start both clients again, qpid memory
> consumption will instantly increase. However, if I decide to read the
> message (so that the message depth stay close to 0), the memory
> consumption will stay low whatever happens.
>
>
>
> I can reproduce this behaviour with both qpid 0.5 and qpid 0.6, with
> durable and non-durable queues on 2 different servers running ubuntu.
> However, on a windows server qpid releases the memory instantly when I
> kill both clients.
>
>
>
> I also see some similar behaviour when running perftests:
>
>
>
> After running "./perftest --size 64 --count 1000000", qpid consumes
> 168Mb of memory (463MB virtual)
>
> After running "./perftest --size 64 --count 1000000 -qt 10", qpid
> consumes 832Mb of memory (1637MB virtual)
>

I don't see the same results as you, if I run perftest in a loop qpidd remains 
stable at about 400Mb virtual memory (this is on latest trunk). If you run 
perftest repeatedly do you see continuous growth over time? Note that freed 
memory is not immediately returned to the OS, so some growth in qpidd size is to 
be expected even if all messages have been removed from queues. However if you 
see continuous growth over time then there probably is a leak to be investigated.


$ while ps up `pgrep qpidd`; do qpid-perftest -s --size 64 --count 1000000 ; done
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 37.2  0.3 401544 52256 ?        Ssl  12:59   1:26 qpidd --auth=no -d
43269.3 42904.7 86263.5 5.26511
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 50.2  0.2 398156 37440 ?        Ssl  12:59   2:07 qpidd --auth=no -d
40394.2 40248.7 80641.5 4.92196
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 61.8  0.2 397716 37856 ?        Ssl  12:59   2:52 qpidd --auth=no -d
41445.9 41308.1 82771.1 5.05194
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 70.9  0.2 395784 37304 ?        Ssl  12:59   3:35 qpidd --auth=no -d
40487.1 40336.8 80821.4 4.93294
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 79.1  0.2 395784 37304 ?        Ssl  12:59   4:20 qpidd --auth=no -d
40717.5 40552.7 81255.8 4.95946
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 85.9  0.2 395964 35260 ?        Ssl  12:59   5:04 qpidd --auth=no -d
40110.3 40026   80195.4 4.89474
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 92.1  0.2 396868 37332 ?        Ssl  12:59   5:49 qpidd --auth=no -d
41388.6 41231.4 82746.6 5.05045
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902 97.5  0.2 395784 37840 ?        Ssl  12:59   6:32 qpidd --auth=no -d
42590   42440   85032.8 5.18999
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
aconway  10902  101  0.2 396824 38880 ?        Ssl  12:59   7:14 qpidd --auth=no -d

P  Please consider your environmental responsibility before printing this email

*********************************************************************************
Ce message peut contenir des informations confidentielles. 
Les idees et opinions presentees dans ce message 
sont celles de son auteur, et ne representent pas necessairement
celles du groupe ABC arbitrage.
Au cas ou il ne vous serait pas destine,
merci d'en aviser l'expediteur immediatement et de le supprimer.

This message may contain confidential information. 
Any views or opinions presented are solely those of its author 
and do not necessarily represent those of ABC arbitrage. 
If you are not the intended recipient,
please notify the sender immediately and delete it.
*********************************************************************************


---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message