qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Conway <acon...@redhat.com>
Subject Bounding proton sender memory, or: on_sendable and PN_TRANSPORT and buffers, Oh My!
Date Thu, 27 Oct 2016 20:35:56 GMT
Cliff has been bugging me about transport events, sending messages and
memory bounding for ages and the penny finally dropped, I think it
deserves a wider audience:

The issue is bounding memory use in a proton sending application. AMQP
flow control (as shown in our examples) covers many/most cases
providing the receiver sets a "reasonable" credit limits.

However, on the sender, if the receiver sets infinite credit, or has a
much bigger notion of "reasonable", proton will buffer messages without
regard to sender constraints. It is quite plausible that
receiver/sender have very different memory constraints - one might be a
large hub server, the other embedded on millions of small devices
(webcams for example)

The `on_sendable` or PN_FLOW event tells you the remote end has given
credit, so you can write a sender that waits for credit before sending.
I think we can use the C PN_TRANSPORT event in a similar way to limit
sender memory. Attached is a C example/explanation.

Some of our language bindings don't expose TRANSPORT and we might want
to think of a more intuitive way to express this. Also TRANSPORT is a
bit of a catch-all event, it does fire when data moves from session to
transport buffers, but it fires for other things too. We might want to
look at the event model.

Meantime I think the attached is a workable approach in C. Would love
to hear comments, this is something we probably should incorporate into
the language bindings
View raw message