qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Sim <g...@redhat.com>
Subject Re: Message auditing in C++ broker
Date Wed, 12 Mar 2014 16:49:43 GMT
On 03/11/2014 08:33 AM, Jakub Scholz wrote:
> Hi Pavel,
>
> This is definitely a feature I would be interested in :-)
>
> Some time ago, I had a brief look at how such feature could be implemented.
> My ideas were going around the *Observer interfaces (not only
> QueueObserver, but also ConnectionObserver etc.) which seem to be able to
> capture most of the important information, not only message enqueues and
> dequeues. It seemed to me that it might be possible to use the observers to
> receive information about new connections, sessions, queues, messages etc.
>
> I know the original issue was mainly about message enqueue and dequeues,
> but adding information about connections, sessions and queue creation would
> create much more complete picture in the audit file. Although I guess to
> some extent that can be already now captured by the object life cycle
> tracking in the regular log files, which can be probably used to complement
> the message audits.
>
> ad 1a)
> Based on my experience with auditing in our internal software applications,
> I would store the messages in whatever way is considered to be the fastest
> for writing (for example as a structures in a binary file?) and provide
> some tools/API for analysis / parsing of the audit file.

I assume the audit log doesn't need to be synced in anyway, i.e. if the 
machine the broker is on has a sudden power failure, its accepted that 
the activity actually recorded on disk may not be fully up to date?

Would the queues being audited generally be durable or might they also 
be transient? (and what about the persistence of messages)?

> Providing the
> audit information as AMQP stream (in a form of replicated queue or as QMF
> messages) seems like something what would create unnecessary overhead to
> the task. For example in my situation, I probably will not be able to
> analyse the messages in realtime anyway due to security reasons. So I would
> still end up consuming such stream in some client and putting it into a
> file for later analysis. Storing it directly into a file would also avoid
> any problems with modifying parts of the messages.

Would tcpdump be a possible route? I.e. capture all the AMQP data in and 
out of the broker, and perhaps then asynchronously process that into a 
more useful form in a database of some kind?

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message