incubator-etch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Marsh <ThomasMa...@gamestop.com>
Subject RE: Etch/C Memory Consumption
Date Thu, 15 Sep 2011 19:41:16 GMT
Hello Michael,

I have started to investigate the memory consumption issue in more detail, and can confirm
it is leaking around 32 bytes per call of say_hello(). That is about the size of the User
helloworld_object and the user->name wchar_t which is allocated in the client loop, but
clearly must be a copy of that data if this is the issue. The leak traceback is below (for
a run with a loop length of 1000):

32,768 bytes in 4 blocks are possibly lost in loss record 567 of 567
    at  malloc (vg_replace_malloc.c:195)
    by apr_pool_create_ex (apr_pools.c:344)
    by new_queue (etch_queue.c:59)
    by new_mailbox_a (etch_plain_mailbox.c:119)
    by new_mailbox (etch_plain_mailbox.c:82)
    by pmboxmgr_transport_call (etch_plain_mailbox_manager.c:543)
    by tcpdelsvc_begincall (etch_transport.c:525)
    by etchremote_begincall (etch_remote.c:129)
    by helloworld_remote_begin_server_say_hello (helloworld_remote_server.c:217)
    by helloworld_remote_server_say_hello (helloworld_remote_server.c:276)
    by main (helloworld_client_main.c:183)
 
The leak size at this location varies depending on the number of times I iterate over the
say_hello() invocation. Here are the leak sizes from this routine at various loop durations:

500 -> 16,384 bytes lost
1,000 -> 32,768 bytes lost
5,000 -> 155,648 bytes lost
10,000 -> 319,488 bytes lost

However, this memory is released upon a call to etch_runtime_shutdown(), so you cannot see
the leak unless you interrupt the program execution (or watch the memory grow...).

As for my platform:
- Linux CentOS 5.4 (32 bit i686)
- libapr-1.4.5
- libapr-util-1.3.12
- libapr-iconv-1.2.1
- etch (latest from SVN)
- gcc 4.1.2

I hope that the traceback may give you some clue as to the source of the leak. Otherwise,
I will dig deeper when I find more time to focus on this again.

Viele Gruesse,

--thomas
 
-----Original Message-----
From: Michael Fitzner [mailto:Michael.Fitzner@bmw-carit.de] 
Sent: Wednesday, September 14, 2011 6:34 AM
To: etch-dev@incubator.apache.org
Subject: AW: Etch/C Memory Consumption

Hi Thomas,
Your code should be correct and no memory leaks in this part. Could you provide me with some
more information about what systems do you use (Linux, Windows) and a call stack of you memory
leak if available. I will also try to reproduce your test.

Thanks
Michael

-----Ursprüngliche Nachricht-----
Von: Thomas Marsh [mailto:ThomasMarsh@gamestop.com]
Gesendet: Dienstag, 13. September 2011 16:47
An: etch-dev@incubator.apache.org
Betreff: RE: Etch/C Memory Consumption

Hello Martijn,

Thanks for your response. However, this is not the source of the leak. There is a clear etch_object_destroy
in the helloworld_remote_begin_server_say_hello() which deallocates any parameters to the
methods (meaning you will get a segfault if you try to reuse the parameters which now no longer
point to valid memory). This usage semantic is also clearly stated in the C Binding notes

>From http://incubator.apache.org/etch/c-binding-tips-tricks.html:

	The C Binding for Etch has the following memory management rules:

	Implementation side: Parameters of functions have to be destroyed by the function implementation
using etch_object_destroy.
	Caller side: Result Objects have to be freed by the caller using etch_object_destroy. Parameters
of calls will be freed by the runtime automatically.

To reiterate, program memory usage in this very simple usage scenario _must_ be stable. I
suspect an ever growing hash table, or some other similar culprit within the runtime. Can
any of the Etch/C binding developers comment?

Thanks, and best regards,

--thomas

-----Original Message-----
From: Martijn Dashorst [mailto:martijn.dashorst@gmail.com]
Sent: Tuesday, September 13, 2011 7:23 AM
To: etch-dev@incubator.apache.org
Subject: Re: Etch/C Memory Consumption

And  user->name = new_stringw(L"User"); will do so as well

Martijn

On Tue, Sep 13, 2011 at 1:52 PM, Martijn Dashorst <martijn.dashorst@gmail.com> wrote:
> user = new_helloworld_user()
>
> will allocate memory for each pass through the loop.
>
> Martijn
>
> On Mon, Sep 12, 2011 at 11:19 PM, Thomas Marsh <ThomasMarsh@gamestop.com> wrote:
>> Hello all,
>>
>> I have a question about memory consumption in the Etch/C runtime based on behavior
we are seeing within our C client. I have modified the C implementation of the HelloWorld
example in the distribution to run an infinite loop of requests:
>>
>> In the main() routine of helloworld_client_main.c:
>>
>>                ...
>>                while (1) {
>>                                user = new_helloworld_user();
>>                                user->id = 5;
>>                                user->name = new_stringw(L"User");
>>                                result = remote->say_hello(remote,

>> user);
>>                                if (is_etch_exception(result)) {
>>                                                ...
>>                                }
>>                                printf("%S\n", result->v.valw);
>>                                etch_object_destroy(result);
>>                }
>>                ...
>>
>> While running this, I see that the memory consumption of the client continually grows.
(In this example, it grows by about 1 mB every 5 seconds.) I cannot see the memory leak when
testing with valgrind, so it would suggest that the leak is in Etch managed memory which is
cleared at exit.
>>
>> My understanding is that the call to remote->say_hello() should delegate responsibility
of deallocation of the parameters to the Etch runtime, and that the client code is only responsible
for deallocating the result object. The memory use should be stable within this tight loop.
Can anyone comment on the potential cause of the memory consumption?
>>
>> Thanks, and best regards,
>>
>> --thomas
>>
>
>
>
> --
> Become a Wicket expert, learn from the best: http://wicketinaction.com
>



--
Become a Wicket expert, learn from the best: http://wicketinaction.com

Mime
View raw message