qpid-proton mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Conway <acon...@redhat.com>
Subject Re: Minimizing the Proton Engine/Messenger RAM Footprint for embedded devices
Date Mon, 28 Apr 2014 17:34:01 GMT
On Fri, 2014-04-25 at 14:18 -0400, Rafael Schloming wrote:
> Hello and welcome to the list.
> 
> See inline for further comments...
> 
> On Fri, Apr 25, 2014 at 11:55 AM, dcjohns41 <david.johnson3@honeywell.com>wrote:
> 
> > Hello Proton Experts,
> >
> > I am new to this mailing list and use of the Proton system, so please do
> > not
> > get offended if I say something naïve.
> >
> 
> No worries. Please don't be shy. ;-)
> 
> 
> > My team is attempting to use the Proton-C Engine/Messenger package on a
> > small footprint embedded device. We are attempting to send sensor
> > information and receive data to/from the MS Azure Service Bus
> > infrastructure.  We are attempting to use the QPid Proton 0.6 release.  The
> > system is using C code on an ARM-xxx processor.
> >
> > Our configuration has very limited RAM and have been attempting to reduce
> > the run-time requirement to fit in <50Kb or so.  Our data needs are not
> > very
> > big and normally would fit in <1024 bytes of payload.
> >
> > We have trimmed the Proton input/output buffers down from 16KB to smaller
> > values.  Have attempted to disable logging and anything else we could to
> > get
> > the footprint smaller.  At the moment we have the RAM consumption near
> > 43Kbytes.  However, once we start attempting to make socket connections, we
> > see additional RAM allocations that push over the 50KB limit.
> >
> > My first question is- has this been done on other systems where the RAM is
> > very limited?  If yes, is that example available for us to review?
> >
> 
> I know it's been used on some embedded platforms, but I'm afraid I don't
> have any details. I'm guessing either the footprint requirements were not
> as small as yours or whatever tuning was required didn't make it back to
> the codebase.
> 
> 
> > Is it feasible to limit some of the functions/options in the library to at
> > least get basic functionality of AMQP 1.0 to Azure/Service Bus running?
> >  Our
> > expectation is that we need to have at least one transmit link and one
> > receive link to pre-defined topics on the Service Bus side.
> >
> 
> There's probably a bunch of different things we could try, but to be honest
> we haven't really taken a serious look at memory utilization. I'd start by
> instrumenting malloc/free so we can identify where the memory usage is
> coming from. I'd actually like to get that kind of instrumentation into the
> Proton codebase so if you are inclined to do this I'd love to see the
> patch. If not I hope to get around to it soon, but its always difficult to
> predict how much time I will have.

I wouldn't instrument the code for this, there are tools that can
measure this kind of thing without code changes.

valgrind has cachegrind which profiles calls in general and can be used
to identify the sources of malloc calls - there's a nice visualization
tool kcahcegrind for looking at the output.

valgrind also has a memory profiler called massif which I haven't really
used.

systemtap provides a general scripting language that you can use to
intercept malloc calls. There are a bunch of memory related scripts
pre-packaged
http://sourceware.org/systemtap/examples/keyword-index.html#MEMORY

There are surely others...

> 
> 
> >
> > Ultimately we need to have AMQPS (via TLS).  But in the short term we are
> > just attempting to get basic communication running.
> >
> 
> The builtin TLS support relies on openssl, so there may be limits to what
> we can do in terms of memory utilization there. Of course you always have
> the option of not using the builtin TLS layer and supplying your own, but
> again I'd say first thing would probably be to instrument and see how close
> we are to your goal.
> 
> --Rafae



Mime
View raw message