activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <rajdav...@gmail.com>
Subject Re: Paging support
Date Tue, 05 Sep 2006 18:28:28 GMT
There will be some changes to the message store api for the following:

getNextMessage(String clientId, String subsName) - next message that  
hasn't been acknowledged
getNextMessage(String clientId, String subsName, MessageId from,  
RecoveryListener) - allow batches of messages to be processed

etc.

In addition the pending LinkedList in Prefetch Subscription has been  
replaced with a PendingMessageCursor interface - to allow different  
implementations to be plugged in.

So a durable subscriber will not have to be recovered - but can  
retrieve messages in batches - taking into account the available  
memory in the message store.

There is a FilePendingMessageCursor - which can be used optionally  
for the temporary offline storage for non-durable Topic subscribers


On 5 Sep 2006, at 18:35, Fateev, Maxim wrote:

> Rob,
> Would you give overview of the proposed design for the paging  
> support in ActiveMQ? The AMQ-845 doesn't provide any specifics.
>
> -----Original Message-----
> From: Rob Davies [mailto:rajdavies@gmail.com]
> Sent: Tuesday, September 05, 2006 10:22 AM
> To: activemq-users@geronimo.apache.org
> Subject: Re: Paging support
>
> A message reference is kept in memory (which consists of a  
> messageId and information like sequence Id to enable faster  
> retrieval from storage).
> There is some work currently in progress in enable paging support  
> in ActiveMQ - see http://issues.apache.org/activemq/browse/AMQ-845
>
>
> On 5 Sep 2006, at 18:07, pradeep wrote:
>
>>
>> Maxim,
>>
>> Can you clarify this
>> Are you storing just the message id or the whole message in memory?
>> Your
>> help will be highly appreciated.
>>
>> Thanks
>> pradeep
>>
>>
>> Fateev, Maxim wrote:
>>>
>>> It would be interesting to see some numbers about memory utilization
>>> per message in backlog.
>>>
>>> IMHO the main problem with keeping references in memory is that it
>>> needs recovery phase on broker startup. It can take a while when
>>> backlog is really big.
>>>
>>> -----Original Message-----
>>> From: James Strachan [mailto:james.strachan@gmail.com]
>>> Sent: Sunday, September 03, 2006 11:51 PM
>>> To: activemq-users@geronimo.apache.org
>>> Subject: Re: Paging support
>>>
>>> Note that MessageReferences != message contents; they are mostly  
>>> just
>>> a MessageId which is pretty small.
>>>
>>> On 8/29/06, Fateev, Maxim <fateev@amazon.com> wrote:
>>>> AFAIK messages are moved from journal to permantent storage
>>>> (JDBC) on
>>>> every checkpoint. The problem is that references for every message
>>>> are still kept in memory.
>>>> IMHO it is inherent limitation of MessageStore API as it is defined
>>>> now.
>>>> The only way to get message is by id. And the only way to have  
>>>> id is
>>>> to keep it in memory.
>>>> I'm looking into changing MessageStore API to one that instead of
>>>>
>>>> Message         getMessage(MessageId identity)
>>>>
>>>> Would provide
>>>>
>>>> Message getNextMessage()
>>>>
>>>> method eliminating need to keep references in memory.
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Anders Bengtsson [mailto:ndrsbngtssn@yahoo.se]
>>>> Sent: Tuesday, August 29, 2006 1:23 AM
>>>> To: activemq-users@geronimo.apache.org
>>>> Subject: Re: Paging support
>>>>
>>>>
>>>>
>>>> gnodet wrote:
>>>>>
>>>>> If you don' t use any persistent store, messages must be kept in
>>>>> memory, so you are obvisouly limited by the available mem.
>>>>>
>>>>
>>>> Yes, this is what I would expect.
>>>>
>>>>
>>>>
>>>>> If you use a jdbc store, messages can be removed from memory and
>>>>> store for later consumption.
>>>>>
>>>>
>>>> Yes, except it doesn't actually work!
>>>>
>>>> It writes things to the store, but it is still very much limited by
>>>> memory for some reason. (Someone reported this as AMQ-845 earlier,
>>>> but with no response).
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Paging-support-tf2163517.html#a6034790
>>>> Sent from the ActiveMQ - User forum at Nabble.com.
>>>>
>>>
>>>
>>> --
>>>
>>> James
>>> -------
>>> http://radio.weblogs.com/0112098/
>>>
>>>
>>
>> --
>> View this message in context: http://www.nabble.com/Paging-support-
>> tf2163517.html#a6156162
>> Sent from the ActiveMQ - User forum at Nabble.com.
>>


Mime
View raw message