activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Smith (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AMQ-674) Composite Destination persisted messages never get cleaned up, halt message producers
Date Tue, 04 Apr 2006 09:11:59 GMT
    [ https://issues.apache.org/activemq/browse/AMQ-674?page=comments#action_35968 ] 

Paul Smith commented on AMQ-674:
--------------------------------

There is a logical queue within our system called "EntityIncrementalIndexer' (don't think
JMS here, just thing in-out pipeline), which updates to mail & docs in our app are sent
as incremental update messages to index nodes.

Each index node, (Index1 & Index2) need to have their own queue, and we use the Composite
Destination feature to clone a message to both destinations:


producer -> JMS -> Index1
                               -> Index2

The producer is configured to know how many nodes in the cluster, and sends queue messages
via composite destinations, at startup it sets aa variable about the index node list, and
creates a Desitaniotn of the form "Index1.EntityIncrementalIndexer,Index2.EntityIncrementalIndex".
 I thought each Index node would then read from it's own queue, with ActiveMQ pushing the
1 sent message into BOTH queues.

If your immediate thought is "Shouldn't this be done via Durable Subscription topics?", you
are probably right, but I've been bitten by vendors Durable Subscription implementation before,
and thought the Composite Destination approach might be simpler, and more elegant.  (wrong!
:))

Now, I'm confused by your comment about RAM.  You see I don't believe any of teh messages
sent since we started using 2 nodes in the composite destination ever got 'missed'.   That
is from our logs I believe that all messages arrived and were consumed by the destination.
 I believe the messages are being persisted, but not marked as consumed.



> Composite Destination persisted messages never get cleaned up, halt message producers
> -------------------------------------------------------------------------------------
>
>          Key: AMQ-674
>          URL: https://issues.apache.org/activemq/browse/AMQ-674
>      Project: ActiveMQ
>         Type: Bug

>   Components: Broker, Message Store
>     Versions: 3.2.1
>     Reporter: Paul Smith
>     Priority: Blocker
>      Fix For: 4.0 RC1

>
>
> well, we've just got bit by something huge.  
> Previously we have been shipping messages from producer to consumer via a named Queue
'Index2.EntityIncrementalIndexer', where the machine index2 has a consumer called "EntityIncrementalIndexer"
using persisted messages.
> The design used Composite destinations so that the destination could be:
> "Index1.EntityIncrementalIndexer,Index2.EntityIncrementalIndexer" 
> and have the 2 hosts get sent the same message simply, and if one goes down, it'll catch
up later.  This gave us the ability to have a mirrored configuration, and go tertiary later
if we want simply by adding a new name in the composite destation.
> With a single name, this works great, been working fine for months.  Yesterday we activated
the _true_ composite destination, and both machines started getting their messages fine. 

> no problems so far, BUT, 12 hours later we have noticed that the broker has now stopped
accepting messages, and a look at the activemq_msgs table shows 33228 messages 

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   https://issues.apache.org/activemq/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message