activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom_Z <tomatgooglec...@arcor.de>
Subject Questions to broker architecture and setup, memory configuration
Date Mon, 31 Mar 2014 18:53:36 GMT
Hi there,

as an ActiveMQ starter I would like you to give me some feedback on my
setup.

Scenario is:
- currently single broker setup with standalone ActiveMQ 5.8 using LevelDB,
JRE 7, Protocol for all connected systems will be OpenWire
- 2 backend systems, let's call them B1 and B2, acting as producer and
consumer in sequential, non-multithreaded working mode
- B1 and B2 do produce different kinds of messages into their own individual
queue (two 'in'-queues from the client perspective)
- these systems can rely on a stable network connection and can be seen as a
fast producer and consumer running all over the day
- on the other side we do have a client application C with up to 250
instances working in parallel as a consumer on the two before mentioned
out-queues
- each client C does produce different kinds of response messages to
different 'out'-queues
- the clients do not have a stable network connection and messages can be
stuck in the queue for a while resulting in large out queues, at least quite
a bundle of messages for some of the consumers can remain for a longer time
- expected data volume: 50.000 messages a day, average size 10kb (range is
from 1kb to 1MB) 
- all messages must be persisted
- all queues and virtual destinations are predefined in activemq.xml

Message processing:
- B1 produces into QIB1 (in-queue for B1) and consumes from QO1, QO2, QO3
(queues-out 1 to 3), spread all over the day 
- B2 usually produces large amounts of messages only once a day into QIB2,
but consumes continiously returning messages from QO4
- every client C consumes request messages designated to that individual
client through JMSXGroupID from QIB1 and QIB2 and produces response messages
to QO1,2,3,4  
- in the opposite direction B1 and B2 are able to process messages for
theirselves delivered to their designated queues
- message order is relevant in both directions (in and out) 
- for testing purposes und to be able to redeliver messages to clients in
case of loss at client side we have defined "Composite Destinations", so
that each message is copied (forwardOnly="false") into an in- respectively
out-archive queue, so that messages can be pushed back into the relevant
queues (we are a little concerned about the quality of implementation on the
client side and finding bugs processing messages in production and like to
be prepared for such cases, archiving will be deactivated later)

1. Is that a suitable setup to use ActiveMQ in production?

2. Is a composite destination from queue to queue okay? In the documentation
often topics are used, from the archive queues will never be consumed - only
copied, deleted or purged and viewed in the queue browser.

3. Is it possible to add an expiry date while forwarding to the archive
queue/topic (Apache Camel is not in use)?

4. One open question is whether to use a single out queue (easier for client
app C) and to use filtered destinations, filtering must then base on a
application specific message property to distribute the message types to the
different out-queues, is that possible? Does it perhaps perform better, on
client and server side?

5. I still have to take a deeper look at the memory settings. The queues do
have quite different characteristics, from being constantly emptied (out
queues) to holding large amounts of messages for lots of clients for some
time being able to deliver at any time (in queues). I guess at least
regarding to both out queues containing messages for up to 250 different
message groups I have to do some finetuning? 

6. What are the side effects of too little memory for queue? Out-queues are
not time or performance critical and B1 and B2 should grab all messages
regardless of the message group, in-queues must deliver fast to the client
according to the requested message group. 

I found a note: "like message selector matching, grouping based on
JMSXGroupID occurs before dispatch on messages in memory. With the default
maxPageSize setting, large backlogs of messages destined for one group they
can block receipt of messages to other groups if they don't all fit in
memory." 

That means if memory is not enough for all(!) groups/messages in the
in-queue, a client possibly receives only a subset of messages for his
message group?

Thanks for any hints and advice,
Tom



--
View this message in context: http://activemq.2283324.n4.nabble.com/Questions-to-broker-architecture-and-setup-memory-configuration-tp4679749.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message