activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: need advice for single queue configuration
Date Tue, 13 Oct 2015 13:44:04 GMT
Andy,

Can you please describe why using a pool of MDBs that each consume one
message at a time till they're done is a problem in TomEE?  It's no problem
in JMS/ActiveMQ (see below), but if TomEE has non-JMS reasons why TomEE
doesn't handle that scenario well (for example, if TomEE intentionally
kills and doesn't restart any MDB that doesn't complete after some
timeout), that would be good to know.

Starving the pool of MDBs doesn't seem like a bad thing if the goal of the
pool is to limit the number of messages being simultaneously consumed.
Obviously you have to accept the consequences of that decision and it will
cause problems in certain scenarios (such as where you have a mix of fast
and slow messages and you want to continue processing fast messages even
when there's a backlog of slow messages, and the fact that your KahaDB data
files might stay around longer than necessary if you're using KahaDB), but
there are other JMS/ActiveMQ constructs available to address those
situations (e.g. priority, selectors).  Acking a message on initial receipt
and then resending it if processing fails is definitely not the recommended
JMS approach, because it modifies the message order on failure (failed
messages go to the back of the line on a resend, but the front on a
redelivery) and you will lose the message entirely if the client crashes
between the ack and the resend.  Instead, use individual ack mode to defer
acknowledging the message until you've processed it successfully.  And if
you let JMS do the redelivery, it will track the redelivery count
automatically.

I'm not saying that a single message consumer feeding a work manager is a
bad paradigm, just that it's not the only one from a JMS perspective, so
unless TomEE has a limitation that makes it not work (and again, if that's
the case, please share details), I don't see a reason to tell Leonardo that
he's using an invalid approach.

Tim
Ouch, I'll try and point you in a better direction. A message consumer
should never block longer than at maximum a few seconds, else you are going
to starve the pool in no time. You should only be checking that the message
payload is valid. If you need a long running task then hand the message off
to a workmanager. If the task fails, then simply resend the message to the
queue and let the next consumer try again. If you do that then be sure to
add some message property to keep count of how many times it's failed, so
the task doesn't keep sending it.

Andy.

agumbrecht@tomitribe.com
http://www.tomitribe.com

Sent from my mobile device.

----- Reply message -----
From: "Tim Bain" <tbain@alumni.duke.edu>
To: "ActiveMQ Users" <users@activemq.apache.org>
Cc: <users@tomee.apache.org>
Subject: need advice for single queue configuration
Date: Fri, Oct 9, 2015 15:20

If you want redelivery to other consumers rather than just the one to which
the message was first delivered (which is how I interpreted your paragraph
about message-driven beans) and you can live with out-of-order delivery,
I'd think you'd want to configure broker redelivery as described at the
bottom of
http://activemq.apache.org/message-redelivery-and-dlq-handling.html.

When TomEE "simply stops consuming the queue", have you taken a thread dump
to see what its threads are doing?  Is it possible that every MDB is in the
middle of processing a 1-hour message?

Have you used a JMX viewer such as JConsole to examine the subscriptions on
your destination on the broker when TomEE is having problems, to see if
it's still subscribed?

Tim
On Sep 11, 2015 11:34 AM, "Leonardo K. Shikida" <shikida@gmail.com> wrote:

> Hi
>
> I need a simple queue.
>
> This queue will be persistent and it will use a database in the backend to
> store the messages.
>
> It will receive produced messages small in size (just one or two string
> attributes) in bursts of 10~100.
>
> It will consume messages by a pool of 100~300 message driven beans
(tomee).
> Each message will be consumed and will trigger a long-running job that may
> take from 1 minute to 1 hour to complete. If the job fails, I want the
> message to return to the queue so it can be consumed again, no matter how
> many retries.
>
> I don't want to send invalid messages to a "poisoned messages queue".
>
> Architecture is simple. The queue will run in a single activeMQ instance
> and consumers are MDBs from a single TomEE+ web application.
>
> So, here's what I have so far
>
> activemq.xml=========================================
>
>     <broker xmlns="http://activemq.apache.org/schema/core"
> brokerName="localhost" dataDirectory="${activemq.data}">
>         <destinationPolicy>
>             <policyMap>
>                 <policyEntries>
>                     <policyEntry topic=">" producerFlowControl="true">
>                         <pendingMessageLimitStrategy>
>                             <constantPendingMessageLimitStrategy
> limit="1000" />
>                         </pendingMessageLimitStrategy>
>                     </policyEntry>
>                     <policyEntry queue=">" producerFlowControl="true"
> memoryLimit="1mb">
>                     </policyEntry>
>                 </policyEntries>
>             </policyMap>
>         </destinationPolicy>
>
>         <persistenceAdapter>
>             <jdbcPersistenceAdapter dataSource="#oracle-ds" />
>         </persistenceAdapter>
>
>         <systemUsage>
>             <systemUsage>
>                 <memoryUsage>
>                     <memoryUsage limit="1 gb" />
>                 </memoryUsage>
>                 <storeUsage>
>                     <storeUsage limit="100 gb" />
>                 </storeUsage>
>                 <tempUsage>
>                     <tempUsage limit="50 gb" />
>                 </tempUsage>
>             </systemUsage>
>         </systemUsage>
>
>         <transportConnectors>
>             <transportConnector name="tcp" uri="tcp://0.0.0.0:61616" />
>         </transportConnectors>
>     </broker>
>
>     <bean id="oracle-ds" class="org.apache.commons.dbcp.BasicDataSource"
> destroy-method="close">
>         <property name="driverClassName" value="oracle.jdbc.OracleDriver"
> />
>         <property name="url" value="jdbc:oracle:thin:@localhost:1521:XE"
> />
>         <property name="username" value="*****" />
>         <property name="password" value="******" />
>         <property name="poolPreparedStatements" value="true" />
>         <property name="maxActive" value="100" />
>     </bean>
>
> tomee.xml=========================================
>
>     <Resource id="Default JMS Resource Adapter"
> type="ActiveMQResourceAdapter">
>         BrokerXmlConfig =  broker:(tcp://localhost:61616)
>         ServerUrl = tcp://localhost:61616
>         ThreadPoolSize = 200
>     </Resource>
>
>     <Resource id="MyJmsConnectionFactory"
> type="javax.jms.ConnectionFactory">
>     ResourceAdapter = Default\ JMS\ Resource\ Adapter
>     PoolMaxSize = 200
>     </Resource>
>
>     <Resource id="JobQueue" type="javax.jms.Queue" />
>
> MDB=========================================
>
> @MessageDriven(activationConfig = {
>         @ActivationConfigProperty(propertyName = "destinationType",
> propertyValue = "javax.jms.Queue"),
>         @ActivationConfigProperty(propertyName = "destination",
> propertyValue = "JobQueue"),
>         @ActivationConfigProperty(propertyName = "maxMessagesPerSessions",
> propertyValue = "1"),
>         @ActivationConfigProperty(propertyName = "maxSessions",
> propertyValue = "200"),
>         @ActivationConfigProperty(propertyName = "acknowledgeMode",
> propertyValue = "Auto-acknowledge") })
>
> =========================================
>
> With this configuration, I am still facing some interruptions.
>
> Sometimes, TomEE simply stops consuming the queue (once or twice per
week),
> so I have to restart it. I am still checking the logs for anything
> suspicious.
>
> I am using TomEE+ 1.6.0 stable (quite old, I know) and a standalone AMQ
> 5.10 stable in a linux environment and oracle JDK 7.
>
> But I'd like some advice on how to properly set up activemq / tomee in
this
> scenario (that's why I am sending this email to both activemq and tomee
> lists)
>
> Thank you very much, any help is welcome.
>
> []
>
> Leo
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message