activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claudio Corsi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-3669) Pending producer with qMirror, messages are not spooled to disk
Date Sun, 20 May 2012 05:20:41 GMT

    [ https://issues.apache.org/jira/browse/AMQ-3669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279665#comment-13279665
] 

Claudio Corsi commented on AMQ-3669:
------------------------------------

I was able to determine what the problem was with this issue.

It is that when we initially forward the message to the mirrored topic.  The message memory
usage instance is set to the topic.

When the message is sent to the queue, the memory usage instance is not set to the queue memory
usage but it still uses the topic memory usage.

The message setRegionDestination method is used to set the message memory usage but this is
only performed if and only if the current message memory usage instance is null.  This is
not the case when forwarding the message to the queue and that is why the memory usage of
the topic gets affected causing the producer flow control to throttle the topic/queue when
that is not the case.

                
> Pending producer with qMirror, messages are not spooled to disk
> ---------------------------------------------------------------
>
>                 Key: AMQ-3669
>                 URL: https://issues.apache.org/jira/browse/AMQ-3669
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker, Message Store
>    Affects Versions: 5.5.1
>         Environment: local test on windows, happens also on linux remotely
>            Reporter: Darius Schier
>
> We fill a queue that is backed by a qmirror with data (persistent) that exceeds the configured
memory limit.
> The producer (producerFlowControl="true" at queue and qmirror) will be throttled when
the the limit is reached, no messages are spooled to disk.
> As an overall result, we are not able to write a lot of messages into amq.
> *Configuration:*
> {code:xml} 
> 	<destinationInterceptors>
> 		<mirroredQueue copyMessage = "true" postfix=".qmirror" prefix=""/>
> 	</destinationInterceptors>
> 	<destinationPolicy>
> 		<policyMap>
> 		  <policyEntries>
> 			<policyEntry topic=">" producerFlowControl="true" memoryLimit="2mb" />
> 			<policyEntry queue="created.static.for.persistent" producerFlowControl="true" memoryLimit="1mb"
/>
> 		  </policyEntries>
> 		</policyMap>
> 	</destinationPolicy> 
> 	<destinations>
> 		<queue physicalName="created.static.for.persistent" />
> 	</destinations>
> 	<persistenceAdapter>
> 		<kahaDB directory="${activemq.base}/data/kahadb"/>
> 	</persistenceAdapter>
> 	
> 	<systemUsage>
> 		<systemUsage>
> 			<memoryUsage><memoryUsage limit="10 mb" /></memoryUsage>
> 			<storeUsage><storeUsage limit="100 mb"/></storeUsage>
> 			<tempUsage><tempUsage limit="100 mb"/></tempUsage>
> 		</systemUsage>
> 	</systemUsage>
> 	
> 	<transportConnectors>
> 		<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
> 	</transportConnectors>
>  {code}
> *Java test code:*
> {code} 
>     ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616");
>     Connection connection = factory.createConnection("user", "pwd");
>     connection.start();
>     Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
>     Destination destination = session.createQueue("created.static.for.persistent");
>     MessageProducer producer = session.createProducer(destination);
>     producer.setDeliveryMode(DeliveryMode.PERSISTENT);
>     char[] m = new char[1024];
>     Arrays.fill(m, 'x');
>     // create some messages that have 1k each
>     for (int i = 0; i < 12000; i++) {
>       ActiveMQTextMessage message = new ActiveMQTextMessage();
>       message.setText(new String(m));
>       producer.send(message);
>     }
>     connection.stop();
>     connection.close();
>   }
> {code} 
> *Expectation:*
> Messages should be written to disk when the memory limit exceeds, all messages should
be available within the queue/topic.
> *Result:*
>  INFO | Usage Manager memory limit (2097152) reached for topic://created.static.for.persistent.qmirror.
Producers will be throttled to the rate at which messages are removed from this destination
to prevent flooding it. See http://activemq.apache.org/producer-flow-control.html for more
info
> Store percent used  : 10 
> Memory percent used : 20
> Temp percent used   : 0
> Interesting: The smaller flow control for the queue (1mb) does not seem to catch but
the qmirror does (2mb).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message