activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bryan Murphy" <bmurphy1...@gmail.com>
Subject Re: ActiveMQ + NMS = Stalled Message Queues
Date Fri, 29 Aug 2008 02:01:22 GMT
I read more about calling acknowledge, and as far as I can tell, because I
was using transactional acknowledgement as well as session commit and
rollback, the call to acknowledge would have been a no-op.
Now, some of my transactions are in fact very long transactions.  We have
two sets of services, one that sits in our data center near our database,
and another one that sits out on Amazon's EC2 cloud processing audio and
video files and syncing them up to S3.  The service by our database handles
messages very quickly and never takes longer than a second (in theory),
however, it often sends out new messages as part of the transaction (which
I'll get to below).

The services out on EC2 can take minutes to execute and hold the transaction
open the whole time.  It may very well be the case that ActiveMQ thinks my
consumers are slow and is throttling them.

The database handlers fail more often than the EC2 handlers.  I found this
odd at first, until I added some performance metrics to our tracing.  It
turns out (on my local machine at least) that establishing a connection can
take upwards of 4.5 seconds.  We establish a new connection and a new
session every single time we send a message out.  Most of the handlers on
our database side send out at least one message, so the transactions are
held open for at least that long.

So, it's obvious to me that there's no connection pooling going on and
reading the FAQ confirms this.  Assuming there was connection pooling was
clearly a bad assumption on my part.  We don't use spring internally, so we
have no pre-built components available to handle connection pooling.  For
now, I'm going to deal with it by simply moving away from transactional
message processing, and immediately acknowledging every message when it is
received.  I'll deal with the consequences of failing message handlers
later.

I was also consuming messages in a very different fashion last time I used a
non-transactional method, so perhaps this change in combination with that
previous change will get my system into working order.  Once again, I'll
post back here with the results of these changes.

If you notice any flaw in my reasoning, please do tell me. :)

Thanks,
Bryan

On Thu, Aug 28, 2008 at 4:30 PM, Bryan Murphy <bmurphy1976@gmail.com> wrote:

> Hi Jim,
>
> We're currently using transactional messages.  What we do is the following:
>
> - create connection factory
> - create session (acknowledgement mode = transactional)
> - create consumer
> - bind handler to consumer.Listener
>
> in the handler we do the following:
>
> - handle message
> - grab reference to session (stored as an instance variable)
> - commit session if sucess, rollback if failure
>
> If we were to switch from transactional to client acknowledge, then I would
> understand calling message.Acknowledge().  If for some reason we failed to
> handle the message, how should we deal with it?  Should we pop it back into
> the queue and then call acknowledge?  Call acknowledge and then pop it back
> into the queue?  Or do something else entirely? :)
>
> Anyway, I'll play around with the acknowledge method and post back here if
> I have any success.  We were originally using "auto acknowledge" and not
> explicitly calling acknowledge on the message before we switched over to
> handling them transactionally.
>

> Thanks,
> Bryan
>
> On Thu, Aug 28, 2008 at 3:42 PM, semog <e.semog@gmail.com> wrote:
>
>>
>> Hi Bryan,
>>
>> I can't get to your source code examples.  My work internet is filtering
>> out
>> that site.  So, I would first check to make sure that you are calling
>> Acknowledge on the received messages.  It may be that the Broker is
>> stopping
>> the send to your clients, because it thinks they are "slow" (i.e., they
>> have
>> not responded to the messages it has already sent to them).
>>
>> Best,
>> Jim
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message