activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jens Bräuer <jens.brae...@numberfour.eu>
Subject Monitoring ActiveMQ queue sizes via StatisticsPlugin
Date Mon, 04 Jul 2011 14:21:03 GMT
Hi everyone,

I am quiet new to ActiveMQ and try to monitor the overall queue size. I found a nice Nagios/Icinga
plugin to do so, which uses the StatisticsPlugin. Currently I am a bit puzzles, because it
seems the same configuration yields different result on different host.
On one host (my VM), everything works as expected, whereas on my EC2-machine I only get a
timeout (no answer at all).

My software-stack reads as follows:
- CentOS 5.6
- Java version "1.6.0_24", Java(TM) SE Runtime Environment (build 1.6.0_24-b07)
- Ruby 1.8.6
- Rubygem-Stomp 1.1.8
- ActiveMQ 5.5

I am pretty sure that I run the same software on both hosts installed from the same RPMs.
I know this is not a "hardcore" ActiveMQ issue, but very appreciate any help.

On both hosts, the configuration is

--------
<beans
  xmlns="http://www.springframework.org/schema/beans"
  xmlns:amq="http://activemq.apache.org/schema/core"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd
  http://activemq.apache.org/camel/schema/spring http://activemq.apache.org/camel/schema/spring/camel-spring.xsd">

    <broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" useJmx="true">
        <managementContext>
            <managementContext createConnector="false"/>
        </managementContext>

        <plugins>
          <statisticsBrokerPlugin/>
          <simpleAuthenticationPlugin>
            <users>
	      	      	 <authenticationUser username="nf-user" password="XXXX" groups="newsfeed-read,newsfeed-write,newsfeed-admin,everyone"/>
	      	      	 <authenticationUser username="mc-admin" password="XXXX" groups="mcollective-read,mcollective-write,mcollective-admin,everyone"/>
	      	      	 <authenticationUser username="mc-user" password="XXXX" groups="mcollective-read,mcollective-write,mcollective-admin,everyone"/>
	      	      	 <authenticationUser username="icinga" password="247fbvbvvn390j4mvebnhb7g4bquwnvjdshkudhdsufssannz"
groups="monitoring-read,monitoring-write,monitoring-admin,everyone"/>
	                  </users>
          </simpleAuthenticationPlugin>
          <authorizationPlugin>
            <map>
              <authorizationMap>
                <authorizationEntries>
                  <authorizationEntry queue=">" write="admins" read="admins" admin="admins"
/>
                  <authorizationEntry topic=">" write="admins" read="admins" admin="admins"
/>
                  <authorizationEntry topic="ActiveMQ.Advisory.>"               read="everyone"
write="everyone" admin="everyone"/>
                  <authorizationEntry queue="ActiveMQ.Statistics.Destination.>" read="monitoring-read"
write="monitoring-write" admin="monitoring-admin" />
                  <authorizationEntry topic="nagios.>"                          read="monitoring-read"
write="monitoring-write" admin="monitoring-admin" />
		  <authorizationEntry topic="mcollective.>" write="mcollective-write" read="mcollective-read"
admin="mcollective-admin" />
		  <authorizationEntry queue="mcollective.>" write="mcollective-write" read="mcollective-read"
admin="mcollective-admin" />
		  <authorizationEntry topic="newsfeed.>" write="newsfeed-write" read="newsfeed-read"
admin="newsfeed-admin" />
		  <authorizationEntry queue="newsfeed.>" write="newsfeed-write" read="newsfeed-read"
admin="newsfeed-admin" />
                </authorizationEntries>
              </authorizationMap>
            </map>
          </authorizationPlugin>
        </plugins>

        <systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage limit="20 mb"/>
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="1 gb" name="foo"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="100 mb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>
          
        <transportConnectors>
            <transportConnector name="openwire" uri="tcp://0.0.0.0:6166"/>
            <transportConnector name="stomp" uri="stomp://0.0.0.0:6163"/>
        </transportConnectors>
    </broker>
</beans>
----------

In the working scenario, the tcpdump trace reads like: 
-------
CONNECT
login:icinga
content-length:0
passcode:247fbvbvvn390j4mvebnhb7g4bquwnvjdshkudhdsufssannz
content-type: text/plain; charset=UTF-8

.CONNECTED
session:ID:centos-vm-51950-1309785600907-3:12

.
SUBSCRIBE
destination:/topic/nagios.statresults.centos-vm
content-length:0
transformation:jms-map-xml
content-type: text/plain; charset=UTF-8

.SEND
reply-to:/topic/nagios.statresults.centos-vm
destination:/queue/ActiveMQ.Statistics.Destination.>
content-length:0
content-type: text/plain; charset=UTF-8

.MESSAGE
message-id:ID:centos-vm-51950-1309785600907-2:1:0:0:23
type:Advisory
transformation:jms-map-xml
destination:/topic/nagios.statresults.centos-vm
timestamp:0
expires:0
priority:0

<map>
  <entry>
    <string>memoryUsage</string>
    <long>0</long>
  </entry>
  <entry>
    <string>dequeueCount</string>
    <long>0</long>
  </entry>
  <entry>
    <string>inflightCount</string>
    <long>0</long>
  </entry>
  <entry>
    <string>messagesCached</string>
    <long>0</long>
  </entry>
  <entry>
    <string>averageEnqueueTime</string>
    <double>0.0</double>
  </entry>
  <entry>
    <string>destinationName</string>
    <string>queue://ActiveMQ.Statistics.Destination.stompcat</string>
  </entry>
  <entry>
    <string>size</string>
    <long>0</long>
  </entry>
  <entry>
    <string>memoryPercentUsage</string>
    <int>0</int>
  </entry>
  <entry>
--------

In the non-working scenario the trace is:

--------
CONNECT
passcode:247fbvbvvn390j4mvebnhb7g4bquwnvjdshkudhdsufssannz
content-length:0
login:icinga
content-type: text/plain; charset=UTF-8

.CONNECTED
session:ID:ip-10-49-29-146-34975-1309785697533-3:43

.
SUBSCRIBE
destination:/topic/nagios.statresults.ip-10-49-29-146
content-length:0
transformation:jms-map-xml
content-type: text/plain; charset=UTF-8

.SEND
reply-to:/topic/nagios.statresults.ip-10-49-29-146
destination:/queue/ActiveMQ.Statistics.Destination.>
content-length:0
content-type: text/plain; charset=UTF-8

.
-------

Notice the difference in the connect-message (field order of passcode/login).

Logfile in the working scenario is:

---------
==> /var/log/activemq/wrapper.log <==
INFO   | jvm 1    | 2011/07/04 16:15:10 | DEBUG MessageDatabase                - Checkpoint
started.
INFO   | jvm 1    | 2011/07/04 16:15:10 | DEBUG MessageDatabase                - Checkpoint
done.

==> /var/log/activemq/activemq.log <==
2011-07-04 16:15:15,073 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
2011-07-04 16:15:15,110 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.
2011-07-04 16:15:15,133 [127.0.0.1:53160] DEBUG TransportConnection            - Setting up
new connection id: ID:centos-vm-51950-1309785600907-3:13, address: /127.0.0.1:53160
2011-07-04 16:15:15,175 [127.0.0.1:53160] DEBUG AbstractRegion                 - localhost
adding consumer: ID:centos-vm-51950-1309785600907-3:13:-1:1 for destination: topic://nagios.statresults.centos-vm
2011-07-04 16:15:15,216 [127.0.0.1:53160] DEBUG TransportConnection            - remove connection
id: ID:centos-vm-51950-1309785600907-3:13
2011-07-04 16:15:15,216 [127.0.0.1:53160] DEBUG AbstractRegion                 - localhost
removing consumer: ID:centos-vm-51950-1309785600907-3:13:-1:1 for destination: topic://nagios.statresults.centos-vm
2011-07-04 16:15:15,217 [.0.0.1:53160-38] DEBUG TransportConnection            - Stopping
connection: /127.0.0.1:53160
2011-07-04 16:15:15,218 [.0.0.1:53160-38] DEBUG TcpTransport                   - Stopping
transport tcp:///127.0.0.1:53160
2011-07-04 16:15:15,218 [.0.0.1:53160-38] DEBUG TransportConnection            - Stopped transport:
/127.0.0.1:53160
2011-07-04 16:15:15,219 [.0.0.1:53160-38] DEBUG TransportConnection            - Connection
Stopped: /127.0.0.1:53160

==> /var/log/activemq/wrapper.log <==
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG MessageDatabase                - Checkpoint
started.
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG MessageDatabase                - Checkpoint
done.
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG TransportConnection            - Setting up
new connection id: ID:centos-vm-51950-1309785600907-3:13, address: /127.0.0.1:53160
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG AbstractRegion                 - localhost
adding consumer: ID:centos-vm-51950-1309785600907-3:13:-1:1 for destination: topic://nagios.statresults.centos-vm
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG TransportConnection            - remove connection
id: ID:centos-vm-51950-1309785600907-3:13
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG AbstractRegion                 - localhost
removing consumer: ID:centos-vm-51950-1309785600907-3:13:-1:1 for destination: topic://nagios.statresults.centos-vm
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG TransportConnection            - Stopping
connection: /127.0.0.1:53160
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG TcpTransport                   - Stopping
transport tcp:///127.0.0.1:53160
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG TransportConnection            - Stopped transport:
/127.0.0.1:53160
INFO   | jvm 1    | 2011/07/04 16:15:15 | DEBUG TransportConnection            - Connection
Stopped: /127.0.0.1:53160
-----------

Whereas in the non-working it is:

-----------
==> /var/log/activemq/wrapper.log <==
INFO   | jvm 1    | 2011/07/04 16:16:58 | DEBUG TransportConnection            - Setting up
new connection id: ID:ip-10-49-29-146-34975-1309785697533-3:70, address: /127.0.0.1:60654
INFO   | jvm 1    | 2011/07/04 16:16:58 | DEBUG AbstractRegion                 - localhost
adding consumer: ID:ip-10-49-29-146-34975-1309785697533-3:70:-1:1 for destination: topic://nagios.statresults.ip-10-49-29-146

==> /var/log/activemq/activemq.log <==
2011-07-04 16:17:00,912 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
started.
2011-07-04 16:17:00,917 [eckpoint Worker] DEBUG MessageDatabase                - Checkpoint
done.

==> /var/log/activemq/wrapper.log <==
INFO   | jvm 1    | 2011/07/04 16:17:01 | DEBUG MessageDatabase                - Checkpoint
started.
INFO   | jvm 1    | 2011/07/04 16:17:01 | DEBUG MessageDatabase                - Checkpoint
done.



==> /var/log/activemq/activemq.log <==
2011-07-04 16:17:03,372 [127.0.0.1:60654] DEBUG Transport                      - Transport
failed: java.io.EOFException
java.io.EOFException
	at java.io.DataInputStream.readByte(DataInputStream.java:250)
	at org.apache.activemq.transport.stomp.StompWireFormat.readLine(StompWireFormat.java:146)
	at org.apache.activemq.transport.stomp.StompWireFormat.parseAction(StompWireFormat.java:162)
	at org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:92)
	at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:227)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
	at java.lang.Thread.run(Thread.java:662)
2011-07-04 16:17:03,373 [0.0.1:60654-366] DEBUG TransportConnection            - Stopping
connection: /127.0.0.1:60654
2011-07-04 16:17:03,373 [0.0.1:60654-366] DEBUG TcpTransport                   - Stopping
transport tcp:///127.0.0.1:60654
2011-07-04 16:17:03,374 [0.0.1:60654-366] DEBUG TransportConnection            - Stopped transport:
/127.0.0.1:60654
2011-07-04 16:17:03,375 [0.0.1:60654-366] DEBUG TransportConnection            - Cleaning
up connection resources: /127.0.0.1:60654
2011-07-04 16:17:03,375 [0.0.1:60654-366] DEBUG TransportConnection            - remove connection
id: ID:ip-10-49-29-146-34975-1309785697533-3:70
2011-07-04 16:17:03,375 [0.0.1:60654-366] DEBUG AbstractRegion                 - localhost
removing consumer: ID:ip-10-49-29-146-34975-1309785697533-3:70:-1:1 for destination: topic://nagios.statresults.ip-10-49-29-146
2011-07-04 16:17:03,376 [0.0.1:60654-366] DEBUG TransportConnection            - Connection
Stopped: /127.0.0.1:60654

==> /var/log/activemq/wrapper.log <==
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG Transport                      - Transport
failed: java.io.EOFException
INFO   | jvm 1    | 2011/07/04 16:17:03 | java.io.EOFException
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at java.io.DataInputStream.readByte(DataInputStream.java:250)
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at org.apache.activemq.transport.stomp.StompWireFormat.readLine(StompWireFormat.java:146)
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at org.apache.activemq.transport.stomp.StompWireFormat.parseAction(StompWireFormat.java:162)
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at org.apache.activemq.transport.stomp.StompWireFormat.unmarshal(StompWireFormat.java:92)
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:227)
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:219)
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:202)
INFO   | jvm 1    | 2011/07/04 16:17:03 | 	at java.lang.Thread.run(Thread.java:662)
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG TransportConnection            - Stopping
connection: /127.0.0.1:60654
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG TcpTransport                   - Stopping
transport tcp:///127.0.0.1:60654
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG TransportConnection            - Stopped transport:
/127.0.0.1:60654
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG TransportConnection            - Cleaning
up connection resources: /127.0.0.1:60654
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG TransportConnection            - remove connection
id: ID:ip-10-49-29-146-34975-1309785697533-3:70
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG AbstractRegion                 - localhost
removing consumer: ID:ip-10-49-29-146-34975-1309785697533-3:70:-1:1 for destination: topic://nagios.statresults.ip-10-49-29-146
INFO   | jvm 1    | 2011/07/04 16:17:03 | DEBUG TransportConnection            - Connection
Stopped: /127.0.0.1:60654
--------------

Thanks a lot for your help,
Jens


Mime
View raw message