activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jahlborn <jahlb...@gmail.com>
Subject Re: Custom code to browse messages in the message store?
Date Mon, 19 Dec 2016 15:06:10 GMT
oh wow!  that worked beautifully!  at first i was trying to deal with
messages that had already been paged in, but then i realized that anything
which has not yet been consumed should still be in the store.  so it really
was as simple as passing a MessageRecoveryListener to the store and I can
browse the entire message list.

thanks for your help!


christopher.l.shannon wrote
> The transaction you are referring to is a transaction on the KahaDB
> index and a new one will be created by KahaDB when calling that method
> so you don't need to worry about it.  In terms of statefulness, you
> are correct that the order index tracks state of where it left off.
> But I believe it is safe to use the recover() method (which always
> resets the cursor and starts at the beginning) as there are things
> like duplicate detection in place so it should be able to handle it.
> 
> In fact, if you call browse() on a Topic from JMX it will eventually
> end up delegating to the recover method on the Topic Store and it will
> return up to the number of messages of the browse page size so it has
> been designed to be called multiple times because it will be called
> every time you call browse().  Here is how it is used and might be a
> good example for basing your code off of:
> https://github.com/apache/activemq/blob/activemq-5.14.2/activemq-broker/src/main/java/org/apache/activemq/broker/region/Topic.java#L638
>  For recoverNextMessages I don't remember if it would work or not
> without taking a closer look.
> 
> Unfortunately I don't think there is really any other methods right
> now to try and iterate over the messages in the store. The best thing
> to do would be to write a quick test and see if it works for you.
> 
> On Fri, Dec 16, 2016 at 3:26 PM, jahlborn &lt;

> jahlborn@

> &gt; wrote:
>> So, digging into the message store recoverNextMessages() code, it's tough
>> to
>> tell what is going in in terms of statefulness.  That method certainly
>> seems
>> like it would get me the information that i'm after.  two concerns: 1. it
>> looks to be expecting an existing Transaction (maybe i already have one
>> in
>> place since i'm working off of an existing session). 2.
>> "recoverNextMessages" seems to imply that it's basing the notion of
>> "next"
>> off of current state and calling this method will change that state.  do
>> i
>> need to do something with the store state to ensure that i don't mess up
>> this state for normal message consumption?
>>
>> thanks for your help so far!
>>
>>
>> christopher.l.shannon wrote
>>> You should be able to implement your own MessageRecoveryListener and
>>> use the recover() or recoverNextMessages() methods on the MessageStore
>>> interface.  The MessageStore object is part of the region Queue (it is
>>> inherited from the parent BaseDestination class)
>>> http://activemq.apache.org/maven/apidocs/org/apache/activemq/store/MessageStore.html
>>>
>>> Calling that method will iterate over each message in the store, load
>>> it into memory and then and pass it to the MessageRecoveryListener
>>> that you provide.  If you are using KahaDB then calling these methods
>>> will lock the index and prevent other writes/reads from the store
>>> while executing the method.
>>>
>>> On Thu, Dec 15, 2016 at 11:09 AM, jahlborn &lt;
>>
>>> jahlborn@
>>
>>> &gt; wrote:
>>>> We have activemq embedded in our product.  Since it runs in the same
>>>> jvm,
>>>> we
>>>> have various resource limits in place.  These limits make the browsing
>>>> functionality somewhat less than ideal (you can only read so many
>>>> messages
>>>> in a large queue).  I was wondering if someone who has deeper knowledge
>>>> of
>>>> activemq could suggest a way to read messages directly from the data
>>>> store
>>>> (or some other way to do unbounded browsing which doesn't use up all
>>>> system
>>>> memory).
>>>>
>>>> I already attempted this a bit myself by adding a method to the region
>>>> Queue
>>>> for first pulling from the "paged in" messages, and then directly from
>>>> the
>>>> "PendingMessageCursor messages" to read messages.  However, despite the
>>>> fact
>>>> that the cursor is backed by the message store, it still seems to want
>>>> to
>>>> "page in" messages, and also abides by the resource limits.  (i was
>>>> borrowing code from Queue.getMessage(String id)).
>>>>
>>>> Is there any way to "bypass" the "paging in" logic and just read
>>>> messages
>>>> directly from the message store?  i'm okay with this being "slow", i'd
>>>> prefer to have a solution which works slowly rather than no solution at
>>>> all...
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://activemq.2283324.n4.nabble.com/Custom-code-to-browse-messages-in-the-message-store-tp4720423.html
>>>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://activemq.2283324.n4.nabble.com/Custom-code-to-browse-messages-in-the-message-store-tp4720423p4720542.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.





--
View this message in context: http://activemq.2283324.n4.nabble.com/Custom-code-to-browse-messages-in-the-message-store-tp4720423p4720593.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message