qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adel Boutros <adelbout...@live.com>
Subject RE: OutOfMemory exception with a Qpid Java Broker linked to qpid dispatcher
Date Fri, 08 Jul 2016 14:48:31 GMT
I forgot to add that when I execute a qdmanage command to delete the "connector" to the broker,
the weird queue disappears.

Adel

> Date: Fri, 8 Jul 2016 07:45:49 -0700
> From: adelboutros@live.com
> To: users@qpid.apache.org
> Subject: Re: OutOfMemory exception with a Qpid Java Broker linked to qpid dispatcher
> 
> Fixing bad formatting on Nabbledue to Hotmail:
> 
> Hello,
> 
> We are doing some performance tests with a Qpid Java Broker connected to a
> Qpid dispatcher.
> We have noticed that after some time, the broker dies with an OutOfMemory
> exception and java heap is dumped.
> After analyzing the heap dump and checking the broker, we have noticed 2
> weird behaviors:
> 
> 1) A queue is created on the broker with a weird Name bound on our topic 
> (perf.topic) with a binding key "#". It will receive all messages. 
> This queue seems to be created only when the dispatcher is present and is
> configured.
> As no consumer is connected to this queue, all messages are kept in-memory
> which causes the OutOfMemory exception in the broker.
> 
> Weird Queue definition
> Name: 18bb86dd-2953-4c6f-8eb2-56e5128963f3
> Type: standard
> State: ACTIVE
> Durable: false
> Lifespan: DELETE_ON_NO_OUTBOUND_LINKS
> Persist Messages: NEVER
> Inbound: 0 msg/s (0.00 B/s)
> Outbound: 0 msg/s (0.00 B/s)
> Size: 2374 msgs (0.00 B)
> Pre-fetched: 0 msgs (0.00 B)
> Oldest Message Age: 356.064 secs
> Enforced Max. Ttl(ms): 0
> Enforced Min. Ttl(ms): 0
> Exclusive: LINK
> 
> Consumer connected to that queue (Is it the dispatcher?): 
> Name: qdlink.OyY41QAJRZ4JGGg
> Mode: MOVE
> 
> 
> 2) All messages even those consumed successfully by the consumers are still
> present on the broker
> 
> From the heap dump, we can see the Berkley DB is keeping a reference to all
> messages. Is this also coming from the above weird queue?
> 
> 
> PS: If we only use the dispatcher instead, we have none of the weird
> behaviors
> 
> Extract from the heap dump (Object holding 
> reference to one of the message header. "Validated" is one the message 
> header fields we set and which is already received by a consumer)
> 
> char[9] @ 0xf59ff3d8  VALIDATED                                                     
            
> '- value java.lang.String @ 0xf59ff3c0  VALIDATED                                   
                  
>   '- value java.util.HashMap$Entry @ 0xf59ff310                                     
                 
>     '- [3] java.util.HashMap$Entry[8] @ 0xf59ff2c0                                  
                 
>       '- table java.util.HashMap @ 0xf59ff188                                       
               
>         '- _appProperties
> org.apache.qpid.server.protocol.v1_0.MessageMetaData_1_0 @ 0xf59fef88               
     
>           '- _metaData
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$MessageDataSoftRef
> @ 0xf59fef70     
>             '- _messageDataRef
> org.apache.qpid.server.store.berkeleydb.AbstractBDBMessageStore$StoredBDBMessage
> @ 0xf59fef20
>               '- _handle org.apache.qpid.server.protocol.v1_0.Message_1_0 @
> 0xf59feef0                         
>                 '- _message
> org.apache.qpid.server.message.AbstractServerMessageImpl$Reference @
> 0xf59621c8          
>                   '- _message
> org.apache.qpid.server.queue.StandardQueueEntry @ 0xf5962188                     
>                     '- _next org.apache.qpid.server.queue.StandardQueueEntry
> @ 0xf5962130                     
>           - this$0
> org.apache.qpid.server.protocol.v1_0.MessageMetaData_1_0$MessageHeader_1_0 @
> 0xf59ff1f0   
>  
> Broker Config
> 1 Virtual Host Node and 1 Virtual Host configured with Berkley DB types (BDB
> with the default configuration).
> 1 Topic (perf.topic)
>      bound to a queue (perfQueue) with a binding key and a jms-selector
> filter
>      bound to an "alternate exchange" of a type fanout which is also bound
> to a queue but with no binding key (empty string)
> 
> Dispatcher Config
> 
> qdmanage -b amqp://localhost:10454 create --type=address prefix=perfQueue
> waypoint=true name=perf.queue.addr
> qdmanage -b amqp://localhost:10454 create --type=address prefix=perf.topic
> waypoint=true name=perf.topic.addr
> qdmanage -b amqp://localhost:10454 create --type=connector
> role=route-container addr=localhost port=10455
> name=localhost.broker.10455.connector
> qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perfQueue
> dir=out connection=localhost.broker.10455.connector
> name=localhost.broker.10455.perfQueue.out
> qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perfQueue
> dir=in connection=localhost.broker.10455.connector
> name=localhost.broker.10455.perfQueue.in
> qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perf.topic
> dir=out connection=localhost.broker.10455.connector
> name=localhost.broker.10455.perf.topic.out
> qdmanage -b amqp://localhost:10454 create --type=autoLink addr=perf.topic
> dir=in connection=localhost.broker.10455.connector
> name=localhost.broker.10455.perf.topic.in
> 
> Clients config
> 3 JMS Consumers connected each to perfQueue on the dispatcher
> 2 JMS producers connected each to perf.topic on the dispatcher
> 
> With the above config, we send a number of messages of which only 1/3 will
> be routed to the "alternate exchange" and never consumed.
> 
> 
> Versions
> Qpid Java Broker: 6.0.0
> Qpid Dispatch: 0.6.0
> JMS: 0.9.0
> 
> Regards,
> Adel
> 
> 
> 
> --
> View this message in context: http://qpid.2158936.n2.nabble.com/OutOfMemory-exception-with-a-Qpid-Java-Broker-linked-to-qpid-dispatcher-tp7647065p7647066.html
> Sent from the Apache Qpid users mailing list archive at Nabble.com.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
> For additional commands, e-mail: users-help@qpid.apache.org
> 
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message