Mailing-List: contact users-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@activemq.apache.org
Received-SPF: pass (nike.apache.org: domain of nhrdls@gmail.com designates
 209.85.220.43 as permitted sender)
Message-ID: <53C5D4EE.8060402@gmail.com>
Date: Tue, 15 Jul 2014 18:27:10 -0700
From: Niranjan Rao <nhrdls@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: users@activemq.apache.org
Subject: How to optimize performance of multiple producers and consumers
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Greetings,

We are on version 5.10 - recently migrated, but don't think my problem 
is version specific.

In short, it boils down to can I evaluate the selector information on 
producer side? I need to make decision whether this is a good time to 
place the job or not.

I have multiple producers and consumers using same queue. Producers can 
be broadly categorized as live and batch consumers.

Between the time producer sends the message and consumer responds back 
with results, the entity in question gets in "locked" state. Since the 
consumers have finite capacity as message processing takes lot of time 
(more than 5 minutes usually), I am trying to avoid locking of entities.

Our consumers already publish heartbeat on a topic which has lot of 
useful data including number of jobs they are running and capacity they 
are configured with. So I have some idea whether I can place the job or not.

The part I can not get around is selectors on the consumers. Consumers 
prioritize messages from live users as long as number of concurrently 
processed messages don't exceed the capacity defined. Batch jobs which 
produce many more messages, however get lower priority.

Given the selector, can I some how evaluate the  selector on producer 
side to check if consumer will process the the job quickly enough?

The consumer capacity is dynamic as operations team can disable some 
consumers or the traffic pattern changes. So asking consumers via hear 
beat seems to be best solution. I don't want consumers to sit idle, nor 
I want to load up the messages and lock my entities for long time. I am 
open to other ideas also

Regards,

Niranjan