qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Godfrey <rob.j.godf...@gmail.com>
Subject Re: Configuring a queue depth limit
Date Wed, 24 Aug 2016 17:45:26 GMT
Hi Ram,

So, firstly 0.28 is pretty old now - it was released at the beginning of
June 2014... how the broker handles messages in memory has changed
considerably since then.  Historically the broker simply used soft
references to manage message memory consumption - that is when the JVM GC
decided that there was insufficient heap memory to store all the message
data, it will evict messages from memory - and the broker will then need to
reload them from disk.  As such I don't think there are any settings in
0.28 which will affect when messages are flowed to disk (note the queue
depth limit setting is merely a trigger for when warning messages are
sent).  There are separate settings which activate producer flow control
(rather than flow to disk) but I don't think this is what you were asking
for.

Later versions of the broker (in particular post v6) take a more pro-active
approach to managing memory, in v6 and above you can set the context
variable "broker.flowToDiskThreshold" which will cause the broker to start
flowing messages to disk once the total memory required by enqueued
messages breaches this threshold.  By default this value is set to 40% of
the available direct memory (note v6 and above store the message data in
direct rather than heap memory).  Individual queues will be assigned
"target" sizes based on their current size so that queues flow to disk in a
somewhat fair manner.

Hope this helps,
Rob

On 24 August 2016 at 17:52, rammohan ganapavarapu <rammohanganap@gmail.com>
wrote:

> Any idea? i also wanted to have those settings at global level of
> virtualhost  as i create queues dynamically using client and i dont know
> the queue name a head of time.
>
>
> Thanks,
> Ram
>
> On Tue, Aug 23, 2016 at 9:30 PM, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
>
> > Hi,
> >
> > I am using java broker 0.28 and would like to know how to configure a
> > queue flow control in initial json config? for example i wanted to store
> > 100MB of messages in memory and when it exceeds flow to disk. From docs i
> > see xml version of config but wanted to see if we have any JSON format.
> >
> > Also if i don't define queue depth limit what is the default settings? it
> > keep jvm heap size worth of messages in queue? what will happened if heap
> > gets full does it flow those messages to disk and processes from disk?
> >
> > Finally how do i decide queue depth settings, based on available RAM or
> > any other criteria?
> >
> > Thanks,
> > RG
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message