qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jakub Scholz <ja...@scholz.cz>
Subject Re: How to enable persistent message store in qpid-cpp-1.35.0
Date Fri, 21 Oct 2016 18:26:05 GMT
Yeah, I know what you are talking about from my own experience. I guess
0.28 might have still used the old message store. It a bit depends on the
exact scenario, but in many cases the linear store caused significant
improvements compared to the old store. It is still not immediate, it can
still take quite a lot of time, but in my use cases it is significantly
better.

J.

On Fri, Oct 21, 2016 at 8:18 PM, rammohan ganapavarapu <
rammohanganap@gmail.com> wrote:

> Jakub,
>
> Thanks for clearing my doubts, yes i wanted to limit the memory usage and
> survive messages on broker restart, in that case i will use option3.
>
> Yes, in java broker will be available right way upon restart and it does
> message recovery in the background. In the 0.28 cpp version we see broker
> will take lot of time to recover if i have lot of messages to recover from
> disk and some times it times out, i hope we new version will have faster
> recovery mechanism.
>
> Ram
>
> On Fri, Oct 21, 2016 at 11:12 AM, Jakub Scholz <jakub@scholz.cz> wrote:
>
> > These are two separate things.
> >
> > The linearstore is the persistent message store. When you create a queue
> as
> > durable (and your queues are durable in the email above) and send
> durable /
> > persistent messages into the queue, the linear store will store all these
> > messages to the disk for recovery in case of broker restart. However, the
> > linear store it self doesn't offload them from memory. So they stay both
> in
> > memory and on disk. When you stop and start the broker, it will read them
> > from the disk into the memory and use them. This is what you are seeing
> > right now.
> >
> > Paging will take the messages from the memory and offload some parts of
> the
> > memory to disk (it will use separate files for that - not the same files
> as
> > the linearstore). Paging it self will only make sure that your memory
> > consumption is under control. But when you stop and start the broker, the
> > paging files are deleted and no message will be recovered from them.
> >
> > Of course you can combine both features and use them together. So
> depending
> > on what is your usecase and what you actually want to achieve you have to
> > configure the broker and the queues, for example:
> > 1) "I want the messages to survive the broker restart, but I have more
> then
> > enough memory" -----> Durable queues / linerstore ON && Paging OFF
> > 2) "I don't need that the messages survive the restart, but I want to use
> > only small amounts of memory for looot of messages" -----> Durable
> queues /
> > Linearstore OFF && Paging ON
> > 3) "I want the messages to survive broker restart and limit the RAM
> usage"
> > -----> Durable queues / Linear store ON && Paging ON.
> >
> > Right now you seem to have the option 1 configured.
> >
> > I'm not sure what does "background recovery" do in Java broker. Does it
> > mean that the broker is available already before loading all the
> messages?
> > In that case no, the C++ broker will always first read all the messages
> > from the disk and only once they are loaded, it will be available for
> > connections. However, the startup is much faster with the linearstore
> than
> > it used to be several releases ago with the old message store.
> >
> > J.
> >
> >
> >
> > On Fri, Oct 21, 2016 at 7:54 PM, rammohan ganapavarapu <
> > rammohanganap@gmail.com> wrote:
> >
> > > Jakub,
> > >
> > > If by default its not flowing to disk, every thing should be in memory
> > > right and if that is the case on broker restart i should loose messages
> > > right? but in my case i do see messages got persisted on broker
> restart.
> > >
> > >
> > > [root@broker1 ~]# ps -ef |grep linearstore
> > > qpidd     1935     1  0 Oct20 ?        00:00:54 /usr/sbin/qpidd
> --config
> > > /etc/qpid/qpidd.conf --daemon --module-dir=/usr/lib64/qpid/daemon/
> > > --load-module=/usr/lib64/qpid/daemon/linearstore.so
> > > --load-module=/usr/lib64/liblinearstoreutils.so --data-dir=/data
> > > --close-fd
> > > 9 --pidfile /var/run/qpidd.pid
> > >
> > > root     21233 21143  0 17:43 pts/0    00:00:00 grep linearstore
> > > [root@broker1 ~]# qpid-stat -q
> > > Queues
> > >   queue                                     dur  autoDel  excl  msg
> > > msgIn  msgOut  bytes  bytesIn  bytesOut  cons  bind
> > >
> > > ============================================================
> > > ==============================================================
> > >   52177446-3a69-4680-b349-d4a5590a6fdd:0.0       Y        Y        0
> > > 0      0       0      0        0         1     2
> > >   ax-q-eaxgroup002-consumer-group-001       Y                   5.47k
> > > 5.67k   196    1.07g  1.11g    38.6m        0     2
> > >   ax-q-eaxgroup002-consumer-group-001-dl    Y                      0
> > > 0      0       0      0        0         0     2
> > >
> > >
> > > [root@broker1 ~]# /etc/init.d/qpidd stop
> > > Stopping Qpid AMQP daemon:                                 [  OK  ]
> > >
> > > [root@eqp042wo ~]# ps -ef |grep linearstore
> > > root     21270 21143  0 17:43 pts/0    00:00:00 grep linearstore
> > >
> > > [root@broker1 ~]# /etc/init.d/qpidd start
> > > Starting Qpid AMQP daemon:                                 [  OK  ]
> > > [root@eqp042wo ~]# ps -ef |grep linearstore
> > > qpidd    21293     1 70 17:43 ?        00:00:21 /usr/sbin/qpidd
> --config
> > > /etc/qpid/qpidd.conf --daemon --module-dir=/usr/lib64/qpid/daemon/
> > > --load-module=/usr/lib64/qpid/daemon/linearstore.so
> > > --load-module=/usr/lib64/liblinearstoreutils.so --data-dir=/data
> > > --close-fd
> > > 9 --pidfile /var/run/qpidd.pid
> > >
> > > root     21325 21143  0 17:44 pts/0    00:00:00 grep linearstore
> > > [root@broker1 ~]# qpid-stat -q
> > > Queues
> > >   queue                                     dur  autoDel  excl  msg
> > > msgIn  msgOut  bytes  bytesIn  bytesOut  cons  bind
> > >
> > > ============================================================
> > > ==============================================================
> > >   a55ce766-c3f3-43ff-9104-c398a5bc6104:0.0       Y        Y        0
> > > 0      0       0      0        0         1     2
> > >   ax-q-eaxgroup002-consumer-group-001       Y                   5.47k
> > > 5.47k     0    1.07g  1.07g       0         0     2
> > >   ax-q-eaxgroup002-consumer-group-001-dl    Y                      0
> > > 0      0       0      0        0         0     2
> > >
> > >
> > > If you observer above, i have 5.47k messages in queue before and after
> > > broker restart if they are not flowing to disk and keeping them only in
> > > RAM, on broker restart i those queues and messages should go away
> right?
> > am
> > > i missing any thing here?. Below is my queue config looks like.
> > >
> > > [root@broker1 ~]# python26 /usr/bin/qpid-config queues
> > > Queue Name                                Attributes
> > > =================================================================
> > > a98385a7-2be9-45bb-a823-5d996e34222d:0.0  auto-del excl
> > > ax-q-eaxgroup002-consumer-group-001       --durable --file-size=5120
> > > --file-count=64 --max-queue-size=1073741824 --max-queue-count=1000000
> > > --limit-policy=flow-to-disk --argument no-local=False
> > > ax-q-eaxgroup002-consumer-group-001-dl    --durable --file-size=6000
> > > --file-count=4 --max-queue-size=52428800 --max-queue-count=100000
> > > --limit-policy=flow-to-disk --argument no-local=False
> > >
> > > One more question, is  background recovery feature available in cpp
> > broker
> > > as well like in java broker?
> > >
> > > Thanks,
> > > Ram
> > >
> > >
> > > On Fri, Oct 21, 2016 at 12:16 AM, Jakub Scholz <jakub@scholz.cz>
> wrote:
> > >
> > > > I think the docs just list an example of what you might get when
> > running
> > > > qpid-config --help. But your actual qpid-config from 1.35.0 should
> nto
> > > > contain flow-to-disk anymore.
> > > >
> > > > Jakub
> > > >
> > > > On Fri, Oct 21, 2016 at 12:45 AM, rammohan ganapavarapu <
> > > > rammohanganap@gmail.com> wrote:
> > > >
> > > > > From the docs i still see "--limit-policy [none | reject |
> > > flow-to-disk"
> > > > to
> > > > > configure queues but its not supported in V1.35.0?
> > > > >
> > > > >
> > > > > https://qpid.apache.org/releases/qpid-cpp-1.35.0/cpp-
> > > > > broker/book/chapter-Managing-CPP-Broker.html#MgmtC-2B-2B-
> > > Usingqpidconfig
> > > > >
> > > > >     --durable            Queue is durable
> > > > >     --file-count N (8)   Number of files in queue's persistence
> > journal
> > > > >     --file-size  N (24)  File size in pages (64Kib/page)
> > > > >     --max-queue-size N   Maximum in-memory queue size as bytes
> > > > >     --max-queue-count N  Maximum in-memory queue size as a number
> of
> > > > > messages
> > > > >     --limit-policy [none | reject | flow-to-disk | ring |
> > ring-strict]
> > > > >                          Action taken when queue limit is reached:
> > > > >                              none (default) - Use broker's default
> > > policy
> > > > >                              reject         - Reject enqueued
> > messages
> > > > >                              flow-to-disk   - Page messages to disk
> > > > >                              ring           - Replace oldest
> > > > > unacquired message with new
> > > > >                              ring-strict    - Replace oldest
> message,
> > > > > reject if oldest is acquired
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Oct 20, 2016 at 3:20 PM, Jakub Scholz <jakub@scholz.cz>
> > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > Do you have the module installed in the path you mentioned?
If
> yes,
> > > > then
> > > > > > you should be able to load the module using the option
> > > > > > "--load-module=/usr/lib64/qpid/daemon/linearstore.so" or by
> > placing
> > > > > > "load-module=/usr/lib64/qpid/daemon/linearstore.so" into your
> > config
> > > > > file.
> > > > > > However, I believe that usually the store should be loaded by
> > default
> > > > > when
> > > > > > it is installed. That is unless you specified that you want
to
> > start
> > > > the
> > > > > > broker without any modules with the no-module-dir option. If
the
> > > module
> > > > > is
> > > > > > loaded, you should see in your log file something like this:
> > > > > > 2016-10-20 22:15:25 [Store] notice Linear Store: Store module
> > > > > initialized;
> > > > > > store-dir=/var/lib/qpidd/store
> > > > > >
> > > > > > Additionally to loading the store, you need to have the queues
> > > created
> > > > as
> > > > > > durable and send the messages as durable/persistent. Otherwise
> the
> > > > > > queues/messages will not use the persistent message store.
> > > > > >
> > > > > > Flow to disk feature has been removed some time ago. The
> > > functionality
> > > > it
> > > > > > provided (offloading messages from memory to disk) is now
> provided
> > by
> > > > the
> > > > > > queue paging feature. Strangely, I can't find it described
> anywhere
> > > in
> > > > > the
> > > > > > Qpid C++ broker documentation. But you can have a look at my
> answer
> > > in
> > > > > this
> > > > > > thread -
> > > > > > http://qpid.2158936.n2.nabble.com/How-do-I-create-a-queue-
> > > > > > larger-than-available-RAM-td7643861.html
> > > > > > - it describes how to configure it and it should be still valid
> in
> > > > > 1.35.0.
> > > > > >
> > > > > > Regards
> > > > > > Jakub
> > > > > >
> > > > > > On Thu, Oct 20, 2016 at 11:56 PM, rammohan ganapavarapu <
> > > > > > rammohanganap@gmail.com> wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > How to enable persistent message store in qpid-cpp-1.35
do i
> have
> > > to
> > > > > > > install/load "/usr/lib64/qpid/daemon/linearstore.so"? also
how
> > to
> > > > > > > configure
> > > > > > > flow_to_disk policy?
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Ram
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message