qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Stitcher <astitc...@redhat.com>
Subject Re: Message auditing in C++ broker
Date Wed, 12 Mar 2014 15:42:15 GMT
On Mon, 2014-03-10 at 05:27 -0400, Pavel Moravec wrote:
> Hi all,
> I raised QPID-5619 to implement message auditing in C++ broker. As a new feature with
more options of implementation and mainly configuration, I would like to discuss it first.
Quoting the JIRA for discussion:

One possible way to implement this would be to add static trace probes
which expose the relevant information at the relevant point in message
or other processing.

These static trace points are compatible between dtrace (Solaris, MacOS
& FreeBSD) and systemtap (Linux) although the qpid build currently only
supports them on Linux (they are compiled a little differently for
dtrace than for systemtap).

This won't in itself give an audit trail. We would also need to write
systemtap scripts to attach to the broker exe and listen for the probes
and produce the audit trail.

I'm not sure if this is an entirely appropriate use of static trace
points but it seems like an interesting possibility especially as they
have almost no overhead when you are not using them, which is a general
problem with audit systems as they need to be inserted in critical

We currently have some probes in the low level IO subsystem and we could
easily add more in other places to see how useful they might be


> There is a reasonable request to have message enqueue/dequeue audited. It can help troubleshooting
or monitoring situations when the broker is suspected from discarding a message that was sent
to it but never received by a consumer.
> Implementation options as to be discussed in qpid users mailing list:
> 1) Key feature:
> a) where to store/send the information about new message enqueue/dequeue?
>   - store it in a configurable log-file (per queue? per broker?) - problem with format
of binary data there
>   - amend/modify replicated queues for this purpose - though replicated queues are designed
not to replicate messages that are dequeued before they have been replicated
>   - generate QMF event to be sent to "qmf.default.topic/agent.ind.event.org_apache_qpid_broker.<queue_name>.audit.<enqueue|dequeue>"
topic - note this would have to change qpid.subject to route the message properly
>   - have dedicated audit exchange for this purpose (of topic type, I expect) - again,
qpid.subject would have to be changed
> b) how to trigger message auditing:
>   - via QueueObserver (thanks Gordon for idea)
> 2) Options of auditing:
> a) audit enqueue+dequeue events only? or also acquire, release, reject?
> b) it should be possible to track only one from    enqueue | dequeue | acquire   events,
or all of them
> c) event/message should contain:
>   - timestamp
>   - original message (or just its header? or configurable?)
>   - type of event (enqueue/dequeue/..)
>   - identification of consumer acquiring/dequeueing the message (and also producer?)
> 3) Provisioning/managing auditing:
> a) have the possibility to turn on/off auditing per queue (via QMF)
> b) have some x-declare arguments for the auditing when declaring the queue?
> Thanks in advance for your feedback.
> Kind regards,
> Pavel
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org

To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org

View raw message