Mailing-List: contact activemq-users-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: activemq-users@geronimo.apache.org
Received-SPF: pass (herse.apache.org: domain of lists@nabble.com designates
 72.21.53.35 as permitted sender)
Message-ID: <7555664.post@talk.nabble.com>
Date: Sun, 26 Nov 2006 22:59:19 -0800 (PST)
From: mark angrish 2 <mark.angrish@gmail.com>
To: activemq-users@geronimo.apache.org
Subject: starting a consumer on a queue with lots of messages
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit


Hi,

I've had a search around but couldn't find anything that could help.

I currently have around 2 million messages that are persisted in my queue. 
The situation I'm in is that the queue must hold onto 1 months (say 200Gb)
worth of data before I can put consumers on to process the backlog.  I
currently only have the default config file setup.  My consumer is set up to
use XA between activemq 4.0.2 and oracle 10g using atomikos transaction
manager 3 integrated with spring 2.0, which allows me to use message driven
pojos!  I am using the marathon fix (if you search the bug fix area you will
find the patch) for activemq to allow it to work with XA.

My questions:

1. My problem is that when i start active mq with the small 2 million record
scenario, it takes ages to start up.  Is there any way to overcome this poor
startup time?

2. This is actually my big pain point at the moment.  When activemq does
eventually start up with the 2 million messages, I can't use hermes to see
the messages (it seems to take hours and still no result), and i can't see
any processing happening on the xa consumer side to the database.  When used
normally say 10's or 100's of messages, so a steady stream of messages,
everything processs to the database just fine.  I tried tinkering with the
prefetch limit but that didn't seem to help either.  Something that could
say "right, let the consumer take x amount of messages off the queue and
process them with XA" would be awesome here.

3. We have about 6 queues on 6 machines with 24 applications acting as
producers into them (so 4 producers going to each queue).  We have a SAN set
up so that each queue can write to its own storage area on the san (i.e. 6
activemq data directories on the SAN).  If i wanted to set up a store and
forward kind of setup to 2 'central queues' where the consumers of the queue
lie, do i just configure a network connector on the 6 queues to point to the
central queue locations and get those central queues to read from all 6 SAN
data directories?  Or do i actually need 6 consumer queues?  In either case
is it ok for them to point to the SAN to read the message stores,
considering the producers are writing and the consumers are removing?

Any help would be massively appreciated!

cheers,

::mark
-- 
View this message in context: http://www.nabble.com/starting-a-consumer-on-a-queue-with-lots-of-messages-tf2710121.html#a7555664
Sent from the ActiveMQ - User mailing list archive at Nabble.com.