activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: java/lang/OutOfMemoryError
Date Tue, 13 Oct 2009 09:11:27 GMT
Can you post your activemq.xml? It looks like you need to restrict the
memory usage of the broker and have it spool messages to disk when the
memory limits are reached. See some additional detail at
http://activemq.apache.org/javalangoutofmemory.html#java.lang.OutOfMemory-SpoolingMessagestoDisk
*

*
2009/10/12 tejeswara <tejeswara@gmail.com>

>
> Our broker is running on Linux (RHEL4 box) and after 1 or 2 days I see
> below
> error in logs.
> Note that, below values are set in bin/activemq in out test lab:
> if [ -z "$ACTIVEMQ_OPTS" ] ; then
>  ACTIVEMQ_OPTS="-Xmx512M -Dorg.apache.activemq.UseDedicatedTaskRunner=true"
> fi
>
> I tried to stress test broker by creating 200 queues and put 5000 messages
> (each about 1500 bytes size)
> in each of these queues. However I could not see any error.  Below error
> occured when broker is idle.
> From logs, only activity during this idle period I see is 'DEBUG
> InactivityMonitor              - 30177 ms elapsed since last read check'
>
> Can someone help in pointing out what is going wrong?
>
> Note that, OutOfMemoryError is seen in our production even when heap size
> is
> bumped up to  1024 and
> dedicated task runner is set to false. This error usvally occurs after 2
> days broker running without restart.
> if [ -z "$ACTIVEMQ_OPTS" ] ; then
>  ACTIVEMQ_OPTS="-Xmx1024M
> -Dorg.apache.activemq.UseDedicatedTaskRunner=false"
> fi
>
>
> Stack Trace:->
>
> JVMDUMP006I Processing Dump Event "uncaught", detail
> "java/lang/OutOfMemoryError" - Please Wait.
> JVMDUMP007I JVM Requesting Snap Dump using
> '/x/web/STAGE2SC5464/idimqbroker/Snap0001.20091010.015439.16581.trc'
> JVMDUMP010I Snap Dump written to
> /x/web/STAGE2SC5464/idimqbroker/Snap0001.20091010.015439.16581.trc
> JVMDUMP007I JVM Requesting Heap Dump using
> '/x/web/STAGE2SC5464/idimqbroker/heapdump.20091010.015439.16581.phd'
> JVMDUMP010I Heap Dump written to
> /x/web/STAGE2SC5464/idimqbroker/heapdump.20091010.015439.16581.phd
> JVMDUMP007I JVM Requesting Java Dump using
> '/x/web/STAGE2SC5464/idimqbroker/javacore.20091010.015439.16581.txt'
> JVMDUMP010I Java Dump written to
> /x/web/STAGE2SC5464/idimqbroker/javacore.20091010.015439.16581.txt
> JVMDUMP013I Processed Dump Event "uncaught", detail
> "java/lang/OutOfMemoryError".
> Exception in thread "MulticastDiscovery: null" java.lang.OutOfMemoryError:
> heap allocation failed
>        at java.net.PlainDatagramSocketImpl.receive0(Native Method)
>        at
> java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:181)
>        at java.net.DatagramSocket.receive(DatagramSocket.java:724)
>        at
>
> org.apache.activemq.transport.discovery.multicast.MulticastDiscoveryAgent.run(MulticastDiscoveryAgent.java:338)
>        at java.lang.Thread.run(Thread.java:803)
> JVMDUMP006I Processing Dump Event "uncaught", detail
> "java/lang/OutOfMemoryError" - Please Wait.
> JVMDUMP007I JVM Requesting Snap Dump using
> '/x/web/STAGE2SC5464/idimqbroker/Snap0002.20091010.034616.16581.trc'
> JVMDUMP010I Snap Dump written to
> /x/web/STAGE2SC5464/idimqbroker/Snap0002.20091010.034616.16581.trc
> JVMDUMP007I JVM Requesting Heap Dump using
> '/x/web/STAGE2SC5464/idimqbroker/heapdump.20091010.034616.16581.phd'
> JVMDUMP016E Aborting: Cannot create file
> (/x/web/STAGE2SC5464/idimqbroker/heapdump.20091010.034616.16581.phd)
> JVMDUMP012E Error in Heap Dump: binary dump file cannot be written
> JVMDUMP007I JVM Requesting Java Dump using
> '/x/web/STAGE2SC5464/idimqbroker/javacore.20091010.034616.16581.txt'
> UTE503: utsGetTracePointIterator cannot allocate iterator's buffer
> JVMDUMP010I Java Dump written to
> /x/web/STAGE2SC5464/idimqbroker/javacore.20091010.034616.16581.txt
> JVMDUMP013I Processed Dump Event "uncaught", detail
> "java/lang/OutOfMemoryError".
> Exception in thread "MulticastDiscovery: tcp://stage2sc5464.:61616"
> java.lang.OutOfMemoryError: heap allocation failed
>        at java.net.PlainDatagramSocketImpl.receive0(Native Method)
>        at
> java.net.PlainDatagramSocketImpl.receive(PlainDatagramSocketImpl.java:181)
>        at java.net.DatagramSocket.receive(DatagramSocket.java:724)
>        at
>
> org.apache.activemq.transport.discovery.multicast.MulticastDiscoveryAgent.run(MulticastDiscoveryAgent.java:338)
>        at java.lang.Thread.run(Thread.java:803)
> JVMDUMP006I Processing Dump Event "uncaught", detail
> "java/lang/OutOfMemoryError" - Please Wait.
> JVMDUMP007I JVM Requesting Snap Dump using
> '/x/web/STAGE2SC5464/idimqbroker/Snap0003.20091010.034640.16581.trc'
> JVMDUMP010I Snap Dump written to
> /x/web/STAGE2SC5464/idimqbroker/Snap0003.20091010.034640.16581.trc
> JVMDUMP007I JVM Requesting Heap Dump using
> '/x/web/STAGE2SC5464/idimqbroker/heapdump.20091010.034640.16581.phd'
> JVMDUMP016E Aborting: Cannot create file
> (/x/web/STAGE2SC5464/idimqbroker/heapdump.20091010.034640.16581.phd)
> JVMDUMP012E Error in Heap Dump: binary dump file cannot be written
> JVMDUMP007I JVM Requesting Java Dump using
> '/x/web/STAGE2SC5464/idimqbroker/javacore.20091010.034640.16581.txt'
> UTE503: utsGetTracePointIterator cannot allocate iterator's buffer
> JVMDUMP010I Java Dump written to
> /x/web/STAGE2SC5464/idimqbroker/javacore.20091010.034640.16581.txt
> JVMDUMP013I Processed Dump Event "uncaught", detail
> "java/lang/OutOfMemoryError".
> Exception in thread "InactivityMonitor WriteCheck"
> java.lang.OutOfMemoryError
>        at java.security.AccessController.getProtectionDomains(Native
> Method)
>        at
> java.security.AccessController.getContext(AccessController.java:132)
>        at java.lang.Thread.initialize(Thread.java:347)
>        at java.lang.Thread.<init>(Thread.java:278)
>        at java.lang.Thread.<init>(Thread.java:170)
>        at
>
> org.apache.activemq.transport.InactivityMonitor$5.newThread(InactivityMonitor.java:303)
>        at
>
> java.util.concurrent.ThreadPoolExecutor.addThread(ThreadPoolExecutor.java:411)
>        at
>
> java.util.concurrent.ThreadPoolExecutor.addIfUnderMaximumPoolSize(ThreadPoolExecutor.java:463)
>        at
>
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:878)
>        at
>
> org.apache.activemq.transport.InactivityMonitor.writeCheck(InactivityMonitor.java:128)
>        at
>
> org.apache.activemq.transport.InactivityMonitor$2.run(InactivityMonitor.java:103)
>        at
>
> org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
>        at java.util.TimerThread.mainLoop(Timer.java:527)
>  at java.util.TimerThread.run(Timer.java:477)
> Exception in thread "RMI TCP Connection(454)-10.15.26.210"
> java.lang.NoClassDefFoundError: java.io.StreamCorruptedException
>        at
>
> sun.rmi.transport.StreamRemoteCall.getResultStream(StreamRemoteCall.java:177)
>        at
> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:335)
>        at sun.rmi.transport.Transport$1.run(Transport.java:168)
>        at
> java.security.AccessController.doPrivileged(AccessController.java:275)
> java.lang.NoClassDefFoundError: java.rmi.UnmarshalException
>        at
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:506)
>        at
>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.handleRequest(TCPTransport.java:838)
>        at
> sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:233)
>        at
>
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:912)
>        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:358)
>        at java.lang.Thread.run(Thread.java:803)
>        at sun.rmi.transport.DGCImpl_Stub.dirty(DGCImpl_Stub.java:67)
>        at
> sun.rmi.transport.DGCClient$EndpointEntry.makeDirtyCall(DGCClient.java:344)
>        at
> sun.rmi.transport.DGCClient$EndpointEntry.access$1600(DGCClient.java:160)
>        at
>
> sun.rmi.transport.DGCClient$EndpointEntry$RenewCleanThread.run(DGCClient.java:555)
>        at java.lang.Thread.run(Thread.java:803)
> Exception in thread "RMI LeaseChecker" java.lang.NoClassDefFoundError:
> java.rmi.server.Unreferenced
>        at sun.rmi.transport.Target.refSetRemove(Target.java:320)
>        at sun.rmi.transport.Target.vmidDead(Target.java:444)
>        at sun.rmi.transport.DGCImpl.checkLeases(DGCImpl.java:265)
>        at sun.rmi.transport.DGCImpl.access$400(DGCImpl.java:47)
>        at sun.rmi.transport.DGCImpl$LeaseChecker.run(DGCImpl.java:329)
>        at java.lang.Thread.run(Thread.java:803)
> Unhandled exception
> Type=Segmentation error vmState=0x00053fff
> J9Generic_Signal_Number=00000004 Signal_Number=0000000b
> Error_Value=00000000
> Signal_Code=00000001
> Handler1=008F64F0 Handler2=00142477 InaccessibleAddress=00000000
> EDI=00000000 ESI=27E112E0 EAX=00000000 EBX=00000000
> ECX=08B7DF40 EDX=00000000
> EIP=00F97DB4 ES=0000007B DS=C02D007B ESP=00855BB8
> EFlags=00210206 CS=00000073 SS=0000007B EBP=00855BB8
> Module=/x/web/STAGE2SC5464/idimqbroker/java/jre/bin/libj9jit23.so
> Module_base_address=00F09000
>
>
> Method_being_compiled=org/springframework/jms/listener/DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run()V
> Target=2_30_20070524_12771_lHdSMR (Linux 2.6.9-55.ELsmp)
> CPU=x86 (8 logical CPUs) (0x47621d000 RAM)
> JVMDUMP006I Processing Dump Event "gpf", detail "" - Please Wait.
> JVMDUMP007I JVM Requesting System Dump using
> '/x/web/STAGE2SC5464/idimqbroker/core.20091011.220932.16581.dmp'
> JVMDUMP012E Error in System Dump: cannot find core file, check "ulimit -c"
> is set high enough
> JVMDUMP007I JVM Requesting Snap Dump using
> '/x/web/STAGE2SC5464/idimqbroker/Snap0004.20091011.220932.16581.trc'
> JVMDUMP010I Snap Dump written to
> /x/web/STAGE2SC5464/idimqbroker/Snap0004.20091011.220932.16581.trc
> JVMDUMP007I JVM Requesting Java Dump using
> '/tmp/javacore.20091011.220932.16581.txt'
> NULL
> ------------------------------------------------------------------------
> 0SECTION       TITLE subcomponent dump routine
> NULL           ===============================
> 1TISIGINFO     Dump Event "gpf" (00002000) received
> 1TIDATETIME    Date:                 2009/10/11 at 22:10:17
> 1TIFILENAME    Javacore filename:
>  /tmp/javacore.20091011.220932.16581.txt
> NULL
> ------------------------------------------------------------------------
> 0SECTION       GPINFO subcomponent dump routine
> NULL           ================================
> 2XHOSLEVEL     OS Level         : Linux 2.6.9-55.ELsmp
> 2XHCPUS        Processors -
> 3XHCPUARCH       Architecture   : x86
> 3XHNUMCPUS       How Many       : 8
> NULL
>
>
>
> --
> View this message in context:
> http://www.nabble.com/java-lang-OutOfMemoryError-tp25864372p25864372.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message