activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vijesh <vijeshkn...@echidnainc.com>
Subject JMS + JDBC Need your input to design a better system
Date Wed, 18 Apr 2012 11:18:58 GMT
I am trying to design a messaging system with non-persistent, auto
acknowledge and aysnch mode of publishing. The expected  load is really high
as this system primarily going to handle the huge number of social media
profiles that I have harvested in my system, which is nearly 5 million. And
I have a really distributed architecture, where the social activity details
for all those profiles is being stored in one db, and the scoring which I am
generating for all those profiles is happening and getting stores in a
different place. As of today with out JMS I am facing some latency in
getting my scored profile in online, which I think can be get rid off by
using the JMS. SO primarily my producer going to send all profile to a
particular queue, so the social activity consumer going to consume the
message from this Q and pull out the fresh details and update to a DB. Hope
this gives a picture on what I am trying to solve, So Here's my query, and
the concern I have for which seeking a solution.

On the consumer side, in the onMesasage() method I am going to have the DB
interaction to insert/update the message to mysql DB after doing some
processing with the message. For me this is straight forward, the only worry
is the latency in the consumer side with respect to the time taken to
process the business logic, as well as DB update or insert, which I think is
fine and works well when we have message cursor in the broker side. But what
I am worried about is the load which is going to come on the DB side , so
will some one help me to understand how I can enable batching on this
context. What I am looking for is the following.

- The db update/insert should happen only batch wise, how to enable it
- The db batch which I was talking about should be inline with JMS message
batch as well, because the broker should be acknowledged only once this
batch's db interaction has been completed.
- ofcource this will be transacted, so if my batch insertion/updation got
failed, how the re-delivery of the message will happen? 

So please let me know if activemq is already having provisions to handle
this kind of situation? Or I need to make my own implementation to deal with
this ?

Thanks


--
View this message in context: http://activemq.2283324.n4.nabble.com/JMS-JDBC-Need-your-input-to-design-a-better-system-tp4567407p4567407.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message