qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carl Trieloff <cctriel...@redhat.com>
Subject Re: BDB Message store - rationale?
Date Wed, 23 Dec 2009 16:25:28 GMT
On 12/23/2009 08:59 AM, Alan Conway wrote:
> On 12/22/2009 05:50 PM, Rob Springer wrote:
>> Hey all - we have run up against the max open files aspect of using the
>> message store plugin, and would rather not require our users edit
>> limits.conf to be able to run our software (and the fact that we'd just
>> be racing against the scaling up of the number of declared queues is
>> scary), so we were considering potential alternative solutions.
>>
>> The first question that came up was regarding the message store plugin's
>> use of journal files - why is such a large number (8) the default? Why
>> are the queues created per-queue, as opposed to a coarser-grained
>> implementation, etc?
>>
>> Google and mailing list searches didn't reveal a ton of rationale /
>> design discussions, so we figured we'd ask directly...
>>
>> Finally, if this isn't the mailing list to ask this on (is there a Red
>> Hat one for the message store?), could y'all direct me there?
>>
>
> This is the right list, all the relevant people hang out here but 
> maybe on vacation.
>
> I believe the main reason for per-queue journal files is so the queue 
> can be allocated a contiguous block of disk space at the outset which 
> is used as a circular buffer, so disk heads dont have to skip around 
> to write successive messages on the same queue. On your other 
> questions, I'll defer to the author of the store.

It is also for AIO/ aysnc performance and dealing with hardware cache 
effects etc. It allows us to get to >600,000 durable msgs/second/IO 
channel on current hardware by using very efficient write patterns. It 
also allows you to map your storage to multiple LUNs if desired for example.

Carl.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message