qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rammohan ganapavarapu <rammohanga...@gmail.com>
Subject Re: How to enable persistent message store in qpid-cpp-1.35.0
Date Fri, 21 Oct 2016 17:54:12 GMT
Jakub,

If by default its not flowing to disk, every thing should be in memory
right and if that is the case on broker restart i should loose messages
right? but in my case i do see messages got persisted on broker restart.


[root@broker1 ~]# ps -ef |grep linearstore
qpidd     1935     1  0 Oct20 ?        00:00:54 /usr/sbin/qpidd --config
/etc/qpid/qpidd.conf --daemon --module-dir=/usr/lib64/qpid/daemon/
--load-module=/usr/lib64/qpid/daemon/linearstore.so
--load-module=/usr/lib64/liblinearstoreutils.so --data-dir=/data --close-fd
9 --pidfile /var/run/qpidd.pid

root     21233 21143  0 17:43 pts/0    00:00:00 grep linearstore
[root@broker1 ~]# qpid-stat -q
Queues
  queue                                     dur  autoDel  excl  msg
msgIn  msgOut  bytes  bytesIn  bytesOut  cons  bind

==========================================================================================================================
  52177446-3a69-4680-b349-d4a5590a6fdd:0.0       Y        Y        0
0      0       0      0        0         1     2
  ax-q-eaxgroup002-consumer-group-001       Y                   5.47k
5.67k   196    1.07g  1.11g    38.6m        0     2
  ax-q-eaxgroup002-consumer-group-001-dl    Y                      0
0      0       0      0        0         0     2


[root@broker1 ~]# /etc/init.d/qpidd stop
Stopping Qpid AMQP daemon:                                 [  OK  ]

[root@eqp042wo ~]# ps -ef |grep linearstore
root     21270 21143  0 17:43 pts/0    00:00:00 grep linearstore

[root@broker1 ~]# /etc/init.d/qpidd start
Starting Qpid AMQP daemon:                                 [  OK  ]
[root@eqp042wo ~]# ps -ef |grep linearstore
qpidd    21293     1 70 17:43 ?        00:00:21 /usr/sbin/qpidd --config
/etc/qpid/qpidd.conf --daemon --module-dir=/usr/lib64/qpid/daemon/
--load-module=/usr/lib64/qpid/daemon/linearstore.so
--load-module=/usr/lib64/liblinearstoreutils.so --data-dir=/data --close-fd
9 --pidfile /var/run/qpidd.pid

root     21325 21143  0 17:44 pts/0    00:00:00 grep linearstore
[root@broker1 ~]# qpid-stat -q
Queues
  queue                                     dur  autoDel  excl  msg
msgIn  msgOut  bytes  bytesIn  bytesOut  cons  bind

==========================================================================================================================
  a55ce766-c3f3-43ff-9104-c398a5bc6104:0.0       Y        Y        0
0      0       0      0        0         1     2
  ax-q-eaxgroup002-consumer-group-001       Y                   5.47k
5.47k     0    1.07g  1.07g       0         0     2
  ax-q-eaxgroup002-consumer-group-001-dl    Y                      0
0      0       0      0        0         0     2


If you observer above, i have 5.47k messages in queue before and after
broker restart if they are not flowing to disk and keeping them only in
RAM, on broker restart i those queues and messages should go away right? am
i missing any thing here?. Below is my queue config looks like.

[root@broker1 ~]# python26 /usr/bin/qpid-config queues
Queue Name                                Attributes
=================================================================
a98385a7-2be9-45bb-a823-5d996e34222d:0.0  auto-del excl
ax-q-eaxgroup002-consumer-group-001       --durable --file-size=5120
--file-count=64 --max-queue-size=1073741824 --max-queue-count=1000000
--limit-policy=flow-to-disk --argument no-local=False
ax-q-eaxgroup002-consumer-group-001-dl    --durable --file-size=6000
--file-count=4 --max-queue-size=52428800 --max-queue-count=100000
--limit-policy=flow-to-disk --argument no-local=False

One more question, is  background recovery feature available in cpp broker
as well like in java broker?

Thanks,
Ram


On Fri, Oct 21, 2016 at 12:16 AM, Jakub Scholz <jakub@scholz.cz> wrote:

> I think the docs just list an example of what you might get when running
> qpid-config --help. But your actual qpid-config from 1.35.0 should nto
> contain flow-to-disk anymore.
>
> Jakub
>
> On Fri, Oct 21, 2016 at 12:45 AM, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
>
> > From the docs i still see "--limit-policy [none | reject | flow-to-disk"
> to
> > configure queues but its not supported in V1.35.0?
> >
> >
> > https://qpid.apache.org/releases/qpid-cpp-1.35.0/cpp-
> > broker/book/chapter-Managing-CPP-Broker.html#MgmtC-2B-2B-Usingqpidconfig
> >
> >     --durable            Queue is durable
> >     --file-count N (8)   Number of files in queue's persistence journal
> >     --file-size  N (24)  File size in pages (64Kib/page)
> >     --max-queue-size N   Maximum in-memory queue size as bytes
> >     --max-queue-count N  Maximum in-memory queue size as a number of
> > messages
> >     --limit-policy [none | reject | flow-to-disk | ring | ring-strict]
> >                          Action taken when queue limit is reached:
> >                              none (default) - Use broker's default policy
> >                              reject         - Reject enqueued messages
> >                              flow-to-disk   - Page messages to disk
> >                              ring           - Replace oldest
> > unacquired message with new
> >                              ring-strict    - Replace oldest message,
> > reject if oldest is acquired
> >
> >
> >
> > On Thu, Oct 20, 2016 at 3:20 PM, Jakub Scholz <jakub@scholz.cz> wrote:
> >
> > > Hi,
> > >
> > > Do you have the module installed in the path you mentioned? If yes,
> then
> > > you should be able to load the module using the option
> > > "--load-module=/usr/lib64/qpid/daemon/linearstore.so" or by placing
> > > "load-module=/usr/lib64/qpid/daemon/linearstore.so" into your config
> > file.
> > > However, I believe that usually the store should be loaded by default
> > when
> > > it is installed. That is unless you specified that you want to start
> the
> > > broker without any modules with the no-module-dir option. If the module
> > is
> > > loaded, you should see in your log file something like this:
> > > 2016-10-20 22:15:25 [Store] notice Linear Store: Store module
> > initialized;
> > > store-dir=/var/lib/qpidd/store
> > >
> > > Additionally to loading the store, you need to have the queues created
> as
> > > durable and send the messages as durable/persistent. Otherwise the
> > > queues/messages will not use the persistent message store.
> > >
> > > Flow to disk feature has been removed some time ago. The functionality
> it
> > > provided (offloading messages from memory to disk) is now provided by
> the
> > > queue paging feature. Strangely, I can't find it described anywhere in
> > the
> > > Qpid C++ broker documentation. But you can have a look at my answer in
> > this
> > > thread -
> > > http://qpid.2158936.n2.nabble.com/How-do-I-create-a-queue-
> > > larger-than-available-RAM-td7643861.html
> > > - it describes how to configure it and it should be still valid in
> > 1.35.0.
> > >
> > > Regards
> > > Jakub
> > >
> > > On Thu, Oct 20, 2016 at 11:56 PM, rammohan ganapavarapu <
> > > rammohanganap@gmail.com> wrote:
> > >
> > > > Hi,
> > > >
> > > > How to enable persistent message store in qpid-cpp-1.35 do i have to
> > > > install/load "/usr/lib64/qpid/daemon/linearstore.so"? also how to
> > > > configure
> > > > flow_to_disk policy?
> > > >
> > > >
> > > > Thanks,
> > > > Ram
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message