qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kim van der Riet <kim.vdr...@redhat.com>
Subject Re: Stress C++ broker without flow control
Date Tue, 03 Mar 2009 14:04:16 GMT
On Tue, 2009-03-03 at 19:25 +0800, Stephen wrote:
> The message is TextMessage of 1k size body
> I still cannot figure out when / how it dies - in terms of number of total 
> messages, rate, duration, broker queue size (if configuration is possible)
Some background on the store and enqueue capacity issues:

The store is in effect a circular disk buffer which is split amongst
several files (8 default). For data security reasons, the store will not
overwrite any file which still contains enqueued records from a previous
write cycle - even if the enqueued record is right at the end of the

In addition, a totally full journal is a fatal condition because
removing a record (dequeueing) requires the ability to write a dequeue
record. For this reason, the journal imposes an apporx. 80% capacity
limit on enqueues, but dequeues are not affected as these may free up
space to continue enqueueing.

When a persistent message is enqueued, the store looks ahead about 20%
of the total capacity of the journal files and makes sure that there are
no enqueued messages still in the file at that logical position. If
enqueued records exist in that file, then a enqueue capacity exception
is thrown and the enqueue denied. (This should not to be a fatal
condition; it should be possible to enqueue again once the blocking
enqueud record(s) have been dequeued.)

The size of the journal (both number and size of files) may be changed
by using appropriate configuration. However, if records are not being
dequeued properly or in a timely manner, then undequeued records can
become "stuck" and will force an enqueue capacity exception. Make sure
that records are being dequeued as they are being consumed.

There is an auto-expand mode in the works for the store; when enabled,
it will automatically expand the journal by inserting a new file between
the last writable file and the one containing the first blocking
enqueued record. This should be seen not so much as a convenience
feature, but as a last-resort action, as this is a performance-hitting
operation that will block operations on the queue for the time it takes
to create and format a large data file. Ideally the journal should be
sized correctly to begin with.

Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org

View raw message