activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jk@penguinsfan.com (JIRA)" <j...@apache.org>
Subject [jira] Commented: (AMQ-1061) Memory usage jumps > 100% with durable subscription and stays there
Date Sun, 19 Nov 2006 22:07:02 GMT
    [ https://issues.apache.org/activemq/browse/AMQ-1061?page=comments#action_37494 ] 
            
jk@penguinsfan.com commented on AMQ-1061:
-----------------------------------------

Looking closer at this, it appears I goofed and didn't set the 200M in my config file.

However, this is still an issue because it is possible for a client to send a single big message
(or several big messages in succession) and kill the server.

It would be nice if the prefetch/page in could be configured in terms of bytes, not just number
of messages.  Also, some kind of global optimization that looks at the current memory usage
percentage and tries to keep it below some threshold.

My application has some use cases where big messages will be going into the queues, where
the queues usually contain lots of small messages but there can be much larger ones and possible
even streams of larger ones in succession.  It would be great if the server could handle this
gracefully.

> Memory usage jumps > 100% with durable subscription and stays there
> -------------------------------------------------------------------
>
>                 Key: AMQ-1061
>                 URL: https://issues.apache.org/activemq/browse/AMQ-1061
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 4.2.0
>         Environment: Windows XP Professional, JVM 1.5.0_09
>            Reporter: jk@penguinsfan.com
>            Priority: Critical
>             Fix For: 4.2.0
>
>
> Using the store pending cursor, activemq.xml configured to use 200M of RAM and Kaha store.
 JVM started with options -Xms512M -Xmx512M -Xmn100M.  Messages are 10KB in size, prefetch
is default. 
> I am seeing the memory usage immediately jump up really high (>100) as soon as a durable
consumer is activated when a large number of messages are waiting for that consumer. 
> The producer hangs under this situation. 
> If the consumer is killed before memory usage drops to normal, memory usage does not
drop (i.e., the server locks forever).
> Sometimes, it is possible to exhaust the Java heap and actually crash the server.
> Here is the kind of things I'm seeing in the DEBUG log: 
> 2006-11-19 13:51:53,046 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 10, to: 9 
> 2006-11-19 13:51:53,046 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 9, to: 8 
> 2006-11-19 13:51:53,046 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 8, to: 7 
> 2006-11-19 13:51:53,046 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 7, to: 6 
> 2006-11-19 13:51:53,046 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 6, to: 5 
> 2006-11-19 13:51:53,062 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 5, to: 4 
> 2006-11-19 13:51:53,062 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 4, to: 3 
> 2006-11-19 13:51:53,062 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 3, to: 2 
> 2006-11-19 13:51:53,062 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 2, to: 1 
> 2006-11-19 13:51:53,062 [ata File Writer] DEBUG UsageManager                   - Memory
usage change.  from: 1, to: 0 
> 2006-11-19 13:51:53,468 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 58, to: 59 
> 2006-11-19 13:51:53,593 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 59, to: 60 
> 2006-11-19 13:51:53,812 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 60, to: 61 
> 2006-11-19 13:51:53,843 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 61, to: 62 
> 2006-11-19 13:51:53,890 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 62, to: 63 
> 2006-11-19 13:51:54,015 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 63, to: 64 
> 2006-11-19 13:51:54,015 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 64, to: 65 
> 2006-11-19 13:51:54,046 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 65, to: 66 
> 2006-11-19 13:51:54,062 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 66, to: 67 
> 2006-11-19 13:51:54,109 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 67, to: 68 
> 2006-11-19 13:51:54,109 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 68, to: 69 
> 2006-11-19 13:51:54,109 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 69, to: 70 
> 2006-11-19 13:51:54,125 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 70, to: 71 
> 2006-11-19 13:51:54,171 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 71, to: 72 
> 2006-11-19 13:51:54,171 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 72, to: 73 
> 2006-11-19 13:51:54,171 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 73, to: 74 
> 2006-11-19 13:51:54,187 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 74, to: 75 
> 2006-11-19 13:51:54,234 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 75, to: 76 
> 2006-11-19 13:51:54,234 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 76, to: 77 
> 2006-11-19 13:51:54,265 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 77, to: 78 
> 2006-11-19 13:51:54,265 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 78, to: 79 
> 2006-11-19 13:51:54,265 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 79, to: 80 
> 2006-11-19 13:51:54,265 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 80, to: 81 
> 2006-11-19 13:51:54,281 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 81, to: 82 
> 2006-11-19 13:51:54,312 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 82, to: 83 
> 2006-11-19 13:51:54,312 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 83, to: 84 
> 2006-11-19 13:51:54,312 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 84, to: 85 
> 2006-11-19 13:51:54,312 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 85, to: 86 
> 2006-11-19 13:51:54,312 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 86, to: 87 
> 2006-11-19 13:51:54,359 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 87, to: 88 
> 2006-11-19 13:51:54,390 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 88, to: 89 
> 2006-11-19 13:51:54,390 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 89, to: 90 
> 2006-11-19 13:51:54,406 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 90, to: 91 
> 2006-11-19 13:51:54,406 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 91, to: 92 
> 2006-11-19 13:51:54,406 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 92, to: 93 
> 2006-11-19 13:51:54,421 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 93, to: 94 
> 2006-11-19 13:51:54,437 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 94, to: 95 
> 2006-11-19 13:51:54,437 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 95, to: 96 
> 2006-11-19 13:51:54,437 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 96, to: 97 
> 2006-11-19 13:51:54,468 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 97, to: 98 
> 2006-11-19 13:51:54,468 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 98, to: 99 
> 2006-11-19 13:51:54,468 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 99, to: 100 
> 2006-11-19 13:51:54,468 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 100, to: 101 
> 2006-11-19 13:51:54,500 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 101, to: 102 
> 2006-11-19 13:51:54,500 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 102, to: 103 
> 2006-11-19 13:51:54,500 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 103, to: 104 
> 2006-11-19 13:51:54,500 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 104, to: 105 
> 2006-11-19 13:51:54,500 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 105, to: 106 
> 2006-11-19 13:51:54,562 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 106, to: 107 
> 2006-11-19 13:51:54,562 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 107, to: 108 
> 2006-11-19 13:51:54,562 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 108, to: 109 
> 2006-11-19 13:51:54,593 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 109, to: 110 
> 2006-11-19 13:51:54,609 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 110, to: 111 
> 2006-11-19 13:51:54,609 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 111, to: 112 
> 2006-11-19 13:51:54,609 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 112, to: 113 
> 2006-11-19 13:51:54,609 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 113, to: 114 
> 2006-11-19 13:51:54,625 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 114, to: 115 
> 2006-11-19 13:51:54,687 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 115, to: 116 
> 2006-11-19 13:51:54,718 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 116, to: 117 
> 2006-11-19 13:51:54,718 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 117, to: 118 
> 2006-11-19 13:51:54,718 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 118, to: 119 
> 2006-11-19 13:51:54,718 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 119, to: 120 
> 2006-11-19 13:51:54,718 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 120, to: 121 
> 2006-11-19 13:51:54,718 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 121, to: 122 
> 2006-11-19 13:51:54,734 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 122, to: 123 
> 2006-11-19 13:51:54,734 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 123, to: 124 
> 2006-11-19 13:51:54,750 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 124, to: 125 
> 2006-11-19 13:51:54,765 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 125, to: 126 
> 2006-11-19 13:51:54,781 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 126, to: 127 
> 2006-11-19 13:51:54,781 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 127, to: 128 
> 2006-11-19 13:51:54,812 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 128, to: 129 
> 2006-11-19 13:51:54,828 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 129, to: 130 
> 2006-11-19 13:51:54,828 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 130, to: 131 
> 2006-11-19 13:51:54,828 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 131, to: 132 
> 2006-11-19 13:51:54,843 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 132, to: 133 
> 2006-11-19 13:51:54,984 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 133, to: 134 
> 2006-11-19 13:51:54,984 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 134, to: 135 
> 2006-11-19 13:51:54,984 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 135, to: 136 
> 2006-11-19 13:51:55,031 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 136, to: 137 
> 2006-11-19 13:51:55,046 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 137, to: 138 
> 2006-11-19 13:51:55,046 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 138, to: 139 
> 2006-11-19 13:51:55,046 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 139, to: 140 
> 2006-11-19 13:51:55,046 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 140, to: 141 
> 2006-11-19 13:51:55,062 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 141, to: 142 
> 2006-11-19 13:51:55,062 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 142, to: 143 
> 2006-11-19 13:51:55,109 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 143, to: 144 
> 2006-11-19 13:51:55,109 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 144, to: 145 
> 2006-11-19 13:51:55,234 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 145, to: 146 
> 2006-11-19 13:51:55,265 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 146, to: 147 
> 2006-11-19 13:51:55,265 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 147, to: 148 
> 2006-11-19 13:51:55,265 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 148, to: 149 
> 2006-11-19 13:51:55,281 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 149, to: 150 
> 2006-11-19 13:51:55,281 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 150, to: 151 
> 2006-11-19 13:51:55,281 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 151, to: 152 
> 2006-11-19 13:51:55,375 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 152, to: 153 
> 2006-11-19 13:51:55,390 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 153, to: 154 
> 2006-11-19 13:51:55,406 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 154, to: 155 
> 2006-11-19 13:51:55,437 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 155, to: 156 
> 2006-11-19 13:51:55,437 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 156, to: 157 
> 2006-11-19 13:51:55,437 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 157, to: 158 
> 2006-11-19 13:51:55,453 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 158, to: 159 
> 2006-11-19 13:51:55,453 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 159, to: 160 
> 2006-11-19 13:51:55,453 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 160, to: 161 
> 2006-11-19 13:51:55,484 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 161, to: 162 
> 2006-11-19 13:51:55,484 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 162, to: 163 
> 2006-11-19 13:51:55,531 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 163, to: 164 
> 2006-11-19 13:51:55,562 [/127.0.0.1:1999] DEBUG UsageManager                   - Memory
usage change.  from: 164, to: 165

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/activemq/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message