activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dror <dro...@hotmail.com>
Subject Batch messages on Producer oR On consumer side?
Date Mon, 25 Aug 2014 18:40:13 GMT
Hi,

My use case:
We get lots of small size (up to 1K) messages and we need to send them over
http for reporting purposes.
Due to performance consideration, resource utilization and readability we
would like to accumulate messages and send them in a batch of 512K (it can
also be based on number of messages).
*Requirements:*
1.	Gets high traffic volume of messages and send them over http in a batch.
2.	On failed delivery, I need to resend the messages in another time
(Exponential backoff)
3.	No data losses (all messages need to be sent)

For number 3:  I am using an embedded persistent broker (vm:// protocol) on
the client side.
I thought about few options:

1. *Message aggregator on the producer side*
Producer will accumulate messages using StreamMessage object and send them
as one large message.
Disadvantage:
   a.	Multiple producers – so message aggregation on producer side is
somehow problematic
   b.	Persistency, if the system shut down un expectedly I lost the messages

2. *Message aggregator on the consumer side*

Producer will send every single message and consumer will consume the
message into an internal byteBuffer when the buffer reach the 512K , they
are send as a report.

1.	I will appreciate any advice here?
2.	Is there is a built in mechanism for that (maybe using JmsTransaction)

Thanks in advance;
Dror




--
View this message in context: http://activemq.2283324.n4.nabble.com/Batch-messages-on-Producer-oR-On-consumer-side-tp4685013.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message