activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karel Gardas <karel.gar...@centrum.cz>
Subject Re: Reply:Reply:Re: Subscriber losing messages (from topic)
Date Fri, 11 Jan 2013 17:26:28 GMT

Hi,

thanks a lot for this fast help! Indeed, I've been using non-durable 
subscriber so I'm glad this behavior is expected although for me it was 
highly confusing.  Thanks for clarification. Now, when I've switched to 
using durable subscriber I see all the messages are delivered to the 
subscriber reliably, although way much slower than in case of 
non-durable consumer. The ratio is like few minutes versus ~170 minutes. 
Note that I'm using ActiveMQ db on Solaris' tmp file-system so 
effectively it's whole located in RAM.

Now, I'm using stock ActiveMQ 5.7.0 release so I'm also using its own 
default configuration.  Ideally for our application I would expect 
slower consumer to block faster producer in case of reaching broker 
internal memory limit. Is there some option in ActiveMQ configuration 
file to switch this behavior on? I've seen this was discussed on 
http://activemq.apache.org/slow-consumers.html as an option what to do 
in case of slower consumer so I hope it is already implemented. Also I'd 
like to avoid durable consumers as they seems to be much slower and more 
importantly we don't need reconnect capability and delivering of 
messages to reconnected consumer yet.

Thanks a lot!
Karel

On 01/11/13 03:03 AM, SuoNayi wrote:
> Sorry for my inaccurate post.
> The behavior that the broker discards messages for the slow non-durable subscription
> depends on the maximum pending messages of the subscription.
> There is a pending message limit strategy for the subscription called PendingMessageLimitStrategy.
> You may post your broker configuration file for more details.
>
>
>
>
> At 2013-01-11 09:40:38,SuoNayi<suonayi2006@163.com>  wrote:
>> Hi, I assume you're using non-durable subscription.
>> If so the eviction strategy will be involved when the memory consumption of the
>> broker exceeds the memory limit.In your case the producer is faster than the consumer.
>> This cause more and more messages are pending in memory in the broker
>> and eventually the broker has to use the eviction strategy to discard messages
>> for the slow non-durable subscription.The default strategy is oldestMessageEvictionStrategy
>> that will discard the oldest message in memory.You may take a look at
>> http://activemq.apache.org/slow-consumer-handling.html
>> In this case the behavior you observed is expected.
>>
>>
>>
>>
>>
>> At 2013-01-11 04:59:52,"Timothy Bish"<tabish121@gmail.com>  wrote:
>>> On Thu, 2013-01-10 at 21:52 +0100, Karel Gardas wrote:
>>>> Hello,
>>>>
>>>> I'm playing a bit with ActiveMQ 5.7.0 and found that under some
>>>> circumstances my subscriber looks like losing messages which are
>>>> delivered to him from the broker. Honestly speaking I don't know if to
>>>> blame subscriber or broker in this case but while looking into web
>>>> console of the broker, the broker claims the messages are delivered, but
>>>> I don't see them on subscriber. It took me some time to duplicate this
>>>> issue on as simple as possible example, but finally I've been able to a
>>>> bit hack activemq's own demo to duplicate it.
>>>>
>>>> If you'd like to see the same effect as I see now, just go to
>>>> activemq/example/src and add:
>>>>
>>>
>>> You should try to create a unit test to represent what's going on so
>>> that we can take closer look.  Seems odd that the sleep would cause
>>> something to get lost unless there's a TTL set on the messages.
>>>
>>>> try {
>>>> Thread.sleep(1);
>>>> }
>>>> catch (Exception ex) {
>>>> ex.printStackTrace();
>>>> }
>>>>
>>>> into ConsumerTool.java's onMessage method. This will ensure that
>>>> onMessage on consumer will run a little bit slower than producer is able
>>>> to send messages and you will see the effect.
>>>>
>>>> Once done, just run consumer with:
>>>>
>>>> ant consumer -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
>>>>
>>>> and producer with:
>>>>
>>>> ant producer -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
>>>>
>>>> if everything is working well, then both consumer and producer should
>>>> end once delivering 1000000 messages. The problem is that consumer does
>>>> not end due to losing some of the messages somewhere. If this does not
>>>> happen on your box, please increase number of milliseconds on
>>>> Thread.sleep to make onMessage even slower. I'm running this on Solaris
>>>> 11/JDK 1.7 on Xeon E5 2.0 GHz here.
>>>>
>>>> Now, I do have following questions:
>>>>
>>>> - am I right assuming the example above should really deliver all the
>>>> messages to the consumer onMessage method?
>>>>
>>>> - is this already known issue or shall I report it properly to
>>>> ActiveMQ's bug tracking system?
>>>>
>>>> Just to make sure, I've also tried running consumer with ant consumer
>>>> -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
>>>> -Durl="tcp://localhost:61616?jms.prefetchPolicy.queuePrefetch=0" -- but
>>>> it neither helps. At least I've thought pre-fetching might cause this so
>>>> I tried...
>>>>
>>>> BTW: I've seen the same issue but not on demo, but on our own
>>>> application also while running subscriber on top of 5.6.0 and 5.5.0
>>>> releases. I always used 5.7.0 release for broker and producer though.
>>>> I've not seen it on demo as I've not tested this with older releases but
>>>> I would guess the issue will be there too.
>>>>
>>>>
>>>> Thanks a lot for any idea about what's going wrong here.
>>>>
>>>> Karel
>>>> PS: resending as my former subscription to the list looks like stuck in
>>>> the middle of the way. Sorry if it went through already.
>>>
>>> --
>>> Tim Bish
>>> Sr Software Engineer | RedHat Inc.
>>> tim.bish@redhat.com | www.fusesource.com | www.redhat.com
>>> skype: tabish121 | twitter: @tabish121
>>> blog: http://timbish.blogspot.com/
>>>
>


Mime
View raw message