qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From acon...@apache.org
Subject [09/41] qpid-proton git commit: NO-JIRA: [c] advanced documentation - threads, buffering
Date Wed, 04 Jul 2018 15:57:41 GMT
NO-JIRA: [c] advanced documentation - threads, buffering

Added an "advanced" page and 2 new topics - multi-threading and some explanation
about proton buffering.

Project: http://git-wip-us.apache.org/repos/asf/qpid-proton/repo
Commit: http://git-wip-us.apache.org/repos/asf/qpid-proton/commit/caefcc71
Tree: http://git-wip-us.apache.org/repos/asf/qpid-proton/tree/caefcc71
Diff: http://git-wip-us.apache.org/repos/asf/qpid-proton/diff/caefcc71

Branch: refs/heads/go1
Commit: caefcc7158517357c110e045605bdf0f976f0eed
Parents: d28fecf
Author: Alan Conway <aconway@redhat.com>
Authored: Mon Apr 30 13:13:14 2018 -0400
Committer: Alan Conway <aconway@redhat.com>
Committed: Tue May 29 09:15:45 2018 -0400

 c/docs/CMakeLists.txt  |  9 ++++++++-
 c/docs/advanced.md     |  5 +++++
 c/docs/buffering.md    | 48 +++++++++++++++++++++++++++++++++++++++++++++
 c/docs/threads.md      | 42 +++++++++++++++++++++++++++++++++++++++
 c/docs/user.doxygen.in |  3 ++-
 5 files changed, 105 insertions(+), 2 deletions(-)

diff --git a/c/docs/CMakeLists.txt b/c/docs/CMakeLists.txt
index 7756e48..f1f7fb5 100644
--- a/c/docs/CMakeLists.txt
+++ b/c/docs/CMakeLists.txt
@@ -18,10 +18,17 @@
   configure_file (${CMAKE_CURRENT_SOURCE_DIR}/user.doxygen.in
-  add_custom_target (docs-c COMMAND ${DOXYGEN_EXECUTABLE} user.doxygen)
+  file(GLOB docsrc ../include/proton/*.h *.md)
+  add_custom_target (docs-c COMMAND
+    COMMAND ${CMAKE_COMMAND} -E remove_directory html # get rid of old files
+    DEPENDS ${docsrc})
   add_dependencies (docs docs-c)
   # HTML files are generated to ./html - put those in the install.

diff --git a/c/docs/advanced.md b/c/docs/advanced.md
new file mode 100644
index 0000000..6293958
--- /dev/null
+++ b/c/docs/advanced.md
@@ -0,0 +1,5 @@
+## Advanced topics
+* @subpage threads
+* @subpage io_page
+* @subpage buffering

diff --git a/c/docs/buffering.md b/c/docs/buffering.md
new file mode 100644
index 0000000..a938205
--- /dev/null
+++ b/c/docs/buffering.md
@@ -0,0 +1,48 @@
+## Buffering {#buffering}
+AMQP and proton have mechanisms to allow an application to control it's use of memory.
+### Outgoing Data
+The unit of memory control in AMQP is the *session*.
+`pn_session_outgoing_bytes()` tells you the total bytes buffered for all
+outgoing deliveries on all sending links belonging to that session.
+Each call to `pn_link_send()` adds to the session's outgoing byte total.  Each
+time proton writes data to the network it reduces the total.  To control the
+memory used by a session, check `pn_session_outgoing_bytes()` before
+calling `pn_link_send()`. If it is too high, stop sending until the link
+gets a @ref PN_LINK_FLOW event.
+The AMQP protocol allows peers to exchange session limits so they can predict
+their buffering requirements for incoming data (
+`pn_session_set_incoming_capacity()` and
+`pn_session_set_outgoing_window()`). Proton will not exceed those limits when
+sending to or receiving from the peer. However proton does *not* limit the
+amount of data buffered in local memory at the request of the application.  It
+is up to the application to ensure it does not request more memory than it wants
+proton to use.
+#### Priority
+Data written on different links can be interleaved with data from any other link
+on the same connection when sending to the peer. Proton does not make any formal
+guarantee of fairness, and does not enforce any kind of priority when deciding
+how to order frames for sending. Using separate links and/or sessions for
+high-priority messages means their frames *can* be sent before already-buffered
+low-priority frames, but there is no guarantee that they *will*.
+If you need to ensure swift delivery of higher-priority messages on the same
+connection as lower-priority ones, then you should control the amount of data
+buffered by proton, and buffer the backlog of low-priority backlog in your own
+There is no point in letting proton buffer more than the outgoing session limits
+since that's all it can transmit without peer confirmation. You may want to
+buffer less, depending on how you value the trade-off between reducing max
+latency for high-priority messages (smaller buffer) and increasing max
+throughput under load (bigger buffer).
+### Incoming Data
+To Be Done... <!-- FIXME aconway 2018-05-03:  -->

diff --git a/c/docs/threads.md b/c/docs/threads.md
new file mode 100644
index 0000000..8c2113b
--- /dev/null
+++ b/c/docs/threads.md
@@ -0,0 +1,42 @@
+## Multithreading {#threads}
+The @ref pn_proactor allows you to create multi-threaded client and server
+applications. You can call `pn_proactor_run` in multiple threads to create a
+thread pool for proton to handle events and IO.
+The @ref proactor functions are safe to call from any thread. Most other
+functions are **not** safe to call concurrently. The following rules applies:
+objects and events associated with *different* connections can safely be used
+concurrently. Objects and events associated with *the same* connection cannot.
+The proactor helps you to handle connections safely: `pn_proactor_get()` and
+`pn_proactor_wait()` will never return events associated with the same
+connection to different threads concurrently. As long as you process only the
+objects available from the event returned by the proactor, you are in no danger.
+When you need to process objects belonging to a *different* connection (for
+example if you receive a message on one connection and want to send it on
+another) you must use locks and thread safe data structures and the
+`pn_connection_wake()` function.
+### The wake function
+`pn_connection_wake()` can be called from any thread as long as the
+`pn_connection_t` has not been freed. It allows any thread to "wake up" a
+connection by generating a @ref PN\_CONNECTION\_WAKE event for that connection.
+For example, supposed a message arrives on connection A and must be sent on
+connection B. The thread that receives events for A from the proactor can safely
+call `pn_link_recv()` on A's link to decode the @ref message. However it cannot
+call `pn_link_send()` on any link belonging to B.
+The decoded @ref message is independent of any connection, so the thread
+handling A can store the message in a thread-safe data structure, and call
+`pn_connection_wake()` to notify B. Shortly after, some thread will get a @ref
+PN\_CONNECTION\_WAKE event for B, and can retrieve the message and send it
+on B's link.
+@note Be careful to ensure (using a mutex for example) that connection B
+cannot be deleted before connection A calls `pn_connection_wake`. Connections
+are automatically deleted by the proactor after the @ref PN\_TRANSPORT\_CLOSED event

diff --git a/c/docs/user.doxygen.in b/c/docs/user.doxygen.in
index 03324ee..ae6894a 100644
--- a/c/docs/user.doxygen.in
+++ b/c/docs/user.doxygen.in
@@ -53,7 +53,8 @@ WARNINGS                = YES
 # Configuration options related to the input files
 INPUT                   = @CMAKE_SOURCE_DIR@/c/include \
-                          @CMAKE_SOURCE_DIR@/c/examples
+                          @CMAKE_SOURCE_DIR@/c/examples \
+                          @CMAKE_CURRENT_SOURCE_DIR@
 FILE_PATTERNS           = *.h *.md *.dox
 EXCLUDE_PATTERNS        = @CMAKE_SOURCE_DIR@/c/examples/*.c \
                           @CMAKE_SOURCE_DIR@/c/examples/*.h \

To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org
For additional commands, e-mail: commits-help@qpid.apache.org

View raw message