activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominic Sobotka <n...@hush.com>
Subject ActiveMQ Stream Performance?
Date Fri, 28 Aug 2009 20:41:31 GMT

In a system I'm working with, we are considering using JMS for grid
processing, mainly for asynchronous jobs. I'm wondering how far we can push
the envelope and let JMS be the main bus behind everything, even the more
synchronous jobs where time to respond is critical. So, before I go much
further I would like a sanity check.

The overall app is very basic. Clients talk to Managers, which submit jobs
to request queues, handing a requestID back to the clients. 

Workers consume these requests, do relevant DB queries and create binary
results. The result blobs are dumped to a database to be persisted for a
certain period of time. A short message dumped on a result queue notifies
the manager process of availability. Clients poll managers periodically
until the process is complete.

Now... for the synchronous implementation, we'd want to skip the database
entirely. Managers would hold the client port open while submitting items to
the request queue. In their response messages, workers would include the
binary results, skipping the database entirely. 

So - can we use JMS Streams to ensure clients begin to receive data as soon
as it becomes available? eg,

- Worker queries database, receives results and transforms them to XML
- Worker creates a JMS message containing the raw content, streaming 100MB
message to the response queue
- Message appears on response queue even before the worker has finished
streaming the message
- In turn, Manager begins to consume response and pass the stream along to
the client, well before the worker has finished streaming to the broker or
the broker to the manager

Just in case I'm not being clear, I don't want any step to have to "finish"
before the client can start receiving the results. As the manager will be
"co-located" with the response queue via an embedded broker, there are no
more network hops than I would anticipate in a direct of
Client->Manager->Worker->Database.

Thanks for any responses. Feel free to be critical.
-- 
View this message in context: http://www.nabble.com/ActiveMQ-Stream-Performance--tp25196653p25196653.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message