activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chirag Pujara <chiragpuj...@gmail.com>
Subject Ativemq instance will not shutdown gracefully.
Date Thu, 25 Jul 2013 19:50:21 GMT
Hello,

when I try to shutdown activemq instance it throws exception that it cannot
connect to jmxUrl for process. and then kills the process after 30 secs.

Here is stack strace:

14:34:39 # */opt/gwx/apache-activemq-5.8.0/activemq_8300/bin/activemq_8300
stop "-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://127.0.0.1:8300/jmxrmi
"*
dirname_prg: /opt/gwx/apache-activemq-5.8.0/activemq_8300/bin
INFO: Loading '/etc/default/activemq'
INFO: Using java '/usr/bin/java'
INFO: Waiting at least 30 seconds for regular process termination of pid
'3234' :
Java Runtime: Oracle Corporation 1.7.0_25 /usr/java/jdk1.7.0_25/jre
  Heap sizes: current=1004928k  free=999683k  max=1004928k
    JVM args: -Xms1G -Xmx1G
-Djava.util.logging.config.file=logging.properties
-Dactivemq.classpath=/opt/gwx/apache-activemq-5.8.0/activemq_8300/conf;
-Dactivemq.home=/opt/gwx/apache-activemq-5.8.0
-Dactivemq.base=/opt/gwx/apache-activemq-5.8.0/activemq_8300
-Dactivemq.conf=/opt/gwx/apache-activemq-5.8.0/activemq_8300/conf
-Dactivemq.data=/opt/gwx/apache-activemq-5.8.0/activemq_8300/data
Extensions classpath:

[/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib,/opt/gwx/apache-activemq-5.8.0/lib,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/camel,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/optional,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/web,/opt/gwx/apache-activemq-5.8.0/activemq_8300/lib/extra,/opt/gwx/apache-activemq-5.8.0/lib/camel,/opt/gwx/apache-activemq-5.8.0/lib/optional,/opt/gwx/apache-activemq-5.8.0/lib/web,/opt/gwx/apache-activemq-5.8.0/lib/extra]
ACTIVEMQ_HOME: /opt/gwx/apache-activemq-5.8.0
ACTIVEMQ_BASE: /opt/gwx/apache-activemq-5.8.0/activemq_8300
ACTIVEMQ_CONF: /opt/gwx/apache-activemq-5.8.0/activemq_8300/conf
ACTIVEMQ_DATA: /opt/gwx/apache-activemq-5.8.0/activemq_8300/data
..Connecting to pid: 3234
......INFO: failed to resolve jmxUrl for pid:3234, using default JMX url
.......Connecting to JMX URL:
service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
..ERROR: java.lang.RuntimeException: Failed to execute stop task. Reason:
java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
        java.net.ConnectException: Connection refused]
java.lang.RuntimeException: Failed to execute stop task. Reason:
java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
        java.net.ConnectException: Connection refused]
        at
org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:116)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at
org.apache.activemq.console.command.AbstractJmxCommand.execute(AbstractJmxCommand.java:387)
        at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
        at org.apache.activemq.console.Main.main(Main.java:115)
ERROR: java.lang.Exception: java.io.IOException: Failed to retrieve
RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
        java.net.ConnectException: Connection refused]
java.lang.Exception: java.io.IOException: Failed to retrieve RMIServer
stub: javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
        java.net.ConnectException: Connection refused]
        at
org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:117)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at
org.apache.activemq.console.command.AbstractJmxCommand.execute(AbstractJmxCommand.java:387)
        at
org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:150)
        at
org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:57)
        at
org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
        at org.apache.activemq.console.Main.main(Main.java:115)
Caused by: java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
        java.net.ConnectException: Connection refused]
        at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:358)
        at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:268)
        at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:227)
        at
org.apache.activemq.console.command.AbstractJmxCommand.createJmxConnector(AbstractJmxCommand.java:279)
        at
org.apache.activemq.console.command.AbstractJmxCommand.createJmxConnection(AbstractJmxCommand.java:302)
        at
org.apache.activemq.console.command.ShutdownCommand.runTask(ShutdownCommand.java:81)
        ... 11 more
Caused by: javax.naming.ServiceUnavailableException [Root exception is
java.rmi.ConnectException: Connection refused to host: localhost; nested
exception is:
        java.net.ConnectException: Connection refused]
        at
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:118)
        at
com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:203)
        at javax.naming.InitialContext.lookup(InitialContext.java:411)
        at
javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1925)
        at
javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1892)
        at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:275)
        ... 16 more
Caused by: java.rmi.ConnectException: Connection refused to host:
localhost; nested exception is:
        java.net.ConnectException: Connection refused
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
        at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
        at
sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
        at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:340)
        at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
        at
com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:114)
        ... 21 more
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at
java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at
java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at java.net.Socket.connect(Socket.java:528)
        at java.net.Socket.<init>(Socket.java:425)
        at java.net.Socket.<init>(Socket.java:208)
        at
sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
        at
sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
        at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
        ... 26 more
............
INFO: Regular shutdown not successful,  sending SIGKILL to process with pid
'3234'


-----------------

Here is what my activemq_8100 script looks like:

## Figure out the ACTIVEMQ_BASE from the directory this script was run from
PRG="$0"
progname=`basename "$0"`
saveddir=`pwd`
# need this for relative symlinks
dirname_prg=`dirname "$PRG"`
cd "$dirname_prg"
echo "dirname_prg: $dirname_prg"
while [ -h "$PRG" ] ; do
  ls=`ls -ld "$PRG"`
  link=`expr "$ls" : '.*-> \(.*\)$'`
  if expr "$link" : '.*/.*' > /dev/null; then
    PRG="$link"
  else
    PRG=`dirname "$PRG"`"/$link"
  fi
done
ACTIVEMQ_BASE=`dirname "$PRG"`/..
cd "$saveddir"

ACTIVEMQ_BASE=`cd "$ACTIVEMQ_BASE" && pwd`

## Add system properties for this instance here (if needed), e.g
#export ACTIVEMQ_OPTS_MEMORY="-Xms256M -Xmx1G"
#export ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS_MEMORY
-Dorg.apache.activemq.UseDedicatedTaskRunner=true
-Djava.util.logging.config.file=logging.properties"

export
ACTIVEMQ_SUNJMX_CONTROL="-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://
127.0.0.1:8100/jmxrmi"
#
ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote.port=8100 "
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.password.file=${ACTIVEMQ_BASE}/conf/jmx.password"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.access.file=${ACTIVEMQ_BASE}/conf/jmx.access"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote.ssl=false"
#ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START
-Dcom.sun.management.jmxremote"

export ACTIVEMQ_SUNJMX_START=$ACTIVEMQ_SUNJMX_START
export ACTIVEMQ_HOME=/opt/gwx/apache-activemq-5.8.0
export ACTIVEMQ_BASE=$ACTIVEMQ_BASE

${ACTIVEMQ_HOME}/bin/activemq "$@"

-----------------------------

here is my activemq_8100/conf/activemq.xml file:

<?xml version="1.0" encoding="UTF-8"?><beans xmlns="
http://www.springframework.org/schema/beans" xmlns:amq="
http://activemq.apache.org/schema/core" xmlns:xsi="
http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd">

    <!-- Allows us to use system properties as variables in this
configuration file -->
    <bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
        <property name="locations">
            <value>file:${activemq.conf}/credentials.properties</value>
        </property>
    </bean>

    <!--
        The <broker> element is used to configure the ActiveMQ broker.
    -->
    <broker xmlns="http://activemq.apache.org/schema/core"
brokerName="activemq_8100" dataDirectory="${activemq.data}">

        <!--
            For better performances use VM cursor and small memory limit.
            For more information, see:

            http://activemq.apache.org/message-cursors.html

            Also, if your producer is "hanging", it's probably due to
producer flow control.
            For more information, see:
            http://activemq.apache.org/producer-flow-control.html
        -->

        <destinationPolicy>
            <policyMap>
              <policyEntries>
                <policyEntry producerFlowControl="true" topic="&gt;">
                    <!-- The constantPendingMessageLimitStrategy is used to
prevent
                         slow topic consumers to block producers and affect
other consumers
                         by limiting the number of messages that are
retained
                         For more information, see:


http://activemq.apache.org/slow-consumer-handling.html

                    -->
                  <pendingMessageLimitStrategy>
                    <constantPendingMessageLimitStrategy limit="1000"/>
                  </pendingMessageLimitStrategy>
                </policyEntry>
               <policyEntry memoryLimit="1mb" producerFlowControl="true"
queue="&gt;">
                  <!-- Use VM cursor for better latency
                       For more information, see:

                       http://activemq.apache.org/message-cursors.html

                  <pendingQueuePolicy>
                    <vmQueueCursor/>
                  </pendingQueuePolicy>
                  -->
                </policyEntry>
              </policyEntries>
            </policyMap>
        </destinationPolicy>

 <managementContext>
            <managementContext createConnector="false"/>
        </managementContext>

 <persistenceAdapter>
            <kahaDB directory="${activemq.data}/kahadb"/>
        </persistenceAdapter>

<systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage limit="64 mb"/>
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="100 gb"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="50 gb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>

        <!--
            The transport connectors expose ActiveMQ over a given protocol
to
            clients and other brokers. For more information, see:

            http://activemq.apache.org/configuring-transports.html
        -->
        <transportConnectors>
            <!-- DOS protection, limit concurrent connections to 1000 and
frame size to 100MB -->
            <transportConnector name="openwire" uri="tcp://0.0.0.0:
${openwirePort}?maximumConnections=1000&amp;wireformat.maxFrameSize=104857600"/>
            <transportConnector name="amqp" uri="amqp://0.0.0.0:
${amqpPort}?maximumConnections=1000&amp;wireformat.maxFrameSize=104857600"/>
        </transportConnectors>

        <!-- destroy the spring context on shutdown to stop jetty -->
        <shutdownHooks>
            <bean xmlns="http://www.springframework.org/schema/beans"
class="org.apache.activemq.hooks.SpringContextHook"/>
        </shutdownHooks>

    </broker>

    <!--
        Enable web consoles, REST and Ajax APIs and demos

        Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
    -->
    <!-- <import resource="jetty.xml"/> -->

</beans>


I have tried to stop without jmxUrl in in stop command and it had same
result.


thanks,
Chirag

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message