activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karel Gardas <karel.gar...@centrum.cz>
Subject Re: Reply:Reply:Re: Subscriber losing messages (from topic)
Date Fri, 11 Jan 2013 17:29:55 GMT

Oops, sorry for the question below, now I see you've already pointed me 
into the right direction pointing to: 
http://activemq.apache.org/slow-consumer-handling.html

I'm going to test it.

Thanks,
Karel

On 01/11/13 06:26 PM, Karel Gardas wrote:
>
> Hi,
>
> thanks a lot for this fast help! Indeed, I've been using non-durable
> subscriber so I'm glad this behavior is expected although for me it was
> highly confusing. Thanks for clarification. Now, when I've switched to
> using durable subscriber I see all the messages are delivered to the
> subscriber reliably, although way much slower than in case of
> non-durable consumer. The ratio is like few minutes versus ~170 minutes.
> Note that I'm using ActiveMQ db on Solaris' tmp file-system so
> effectively it's whole located in RAM.
>
> Now, I'm using stock ActiveMQ 5.7.0 release so I'm also using its own
> default configuration. Ideally for our application I would expect slower
> consumer to block faster producer in case of reaching broker internal
> memory limit. Is there some option in ActiveMQ configuration file to
> switch this behavior on? I've seen this was discussed on
> http://activemq.apache.org/slow-consumers.html as an option what to do
> in case of slower consumer so I hope it is already implemented. Also I'd
> like to avoid durable consumers as they seems to be much slower and more
> importantly we don't need reconnect capability and delivering of
> messages to reconnected consumer yet.
>
> Thanks a lot!
> Karel
>
> On 01/11/13 03:03 AM, SuoNayi wrote:
>> Sorry for my inaccurate post.
>> The behavior that the broker discards messages for the slow
>> non-durable subscription
>> depends on the maximum pending messages of the subscription.
>> There is a pending message limit strategy for the subscription called
>> PendingMessageLimitStrategy.
>> You may post your broker configuration file for more details.
>>
>>
>>
>>
>> At 2013-01-11 09:40:38,SuoNayi<suonayi2006@163.com> wrote:
>>> Hi, I assume you're using non-durable subscription.
>>> If so the eviction strategy will be involved when the memory
>>> consumption of the
>>> broker exceeds the memory limit.In your case the producer is faster
>>> than the consumer.
>>> This cause more and more messages are pending in memory in the broker
>>> and eventually the broker has to use the eviction strategy to discard
>>> messages
>>> for the slow non-durable subscription.The default strategy is
>>> oldestMessageEvictionStrategy
>>> that will discard the oldest message in memory.You may take a look at
>>> http://activemq.apache.org/slow-consumer-handling.html
>>> In this case the behavior you observed is expected.
>>>
>>>
>>>
>>>
>>>
>>> At 2013-01-11 04:59:52,"Timothy Bish"<tabish121@gmail.com> wrote:
>>>> On Thu, 2013-01-10 at 21:52 +0100, Karel Gardas wrote:
>>>>> Hello,
>>>>>
>>>>> I'm playing a bit with ActiveMQ 5.7.0 and found that under some
>>>>> circumstances my subscriber looks like losing messages which are
>>>>> delivered to him from the broker. Honestly speaking I don't know if to
>>>>> blame subscriber or broker in this case but while looking into web
>>>>> console of the broker, the broker claims the messages are
>>>>> delivered, but
>>>>> I don't see them on subscriber. It took me some time to duplicate this
>>>>> issue on as simple as possible example, but finally I've been able
>>>>> to a
>>>>> bit hack activemq's own demo to duplicate it.
>>>>>
>>>>> If you'd like to see the same effect as I see now, just go to
>>>>> activemq/example/src and add:
>>>>>
>>>>
>>>> You should try to create a unit test to represent what's going on so
>>>> that we can take closer look. Seems odd that the sleep would cause
>>>> something to get lost unless there's a TTL set on the messages.
>>>>
>>>>> try {
>>>>> Thread.sleep(1);
>>>>> }
>>>>> catch (Exception ex) {
>>>>> ex.printStackTrace();
>>>>> }
>>>>>
>>>>> into ConsumerTool.java's onMessage method. This will ensure that
>>>>> onMessage on consumer will run a little bit slower than producer is
>>>>> able
>>>>> to send messages and you will see the effect.
>>>>>
>>>>> Once done, just run consumer with:
>>>>>
>>>>> ant consumer -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
>>>>>
>>>>> and producer with:
>>>>>
>>>>> ant producer -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
>>>>>
>>>>> if everything is working well, then both consumer and producer should
>>>>> end once delivering 1000000 messages. The problem is that consumer
>>>>> does
>>>>> not end due to losing some of the messages somewhere. If this does not
>>>>> happen on your box, please increase number of milliseconds on
>>>>> Thread.sleep to make onMessage even slower. I'm running this on
>>>>> Solaris
>>>>> 11/JDK 1.7 on Xeon E5 2.0 GHz here.
>>>>>
>>>>> Now, I do have following questions:
>>>>>
>>>>> - am I right assuming the example above should really deliver all the
>>>>> messages to the consumer onMessage method?
>>>>>
>>>>> - is this already known issue or shall I report it properly to
>>>>> ActiveMQ's bug tracking system?
>>>>>
>>>>> Just to make sure, I've also tried running consumer with ant consumer
>>>>> -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
>>>>> -Durl="tcp://localhost:61616?jms.prefetchPolicy.queuePrefetch=0" --
>>>>> but
>>>>> it neither helps. At least I've thought pre-fetching might cause
>>>>> this so
>>>>> I tried...
>>>>>
>>>>> BTW: I've seen the same issue but not on demo, but on our own
>>>>> application also while running subscriber on top of 5.6.0 and 5.5.0
>>>>> releases. I always used 5.7.0 release for broker and producer though.
>>>>> I've not seen it on demo as I've not tested this with older
>>>>> releases but
>>>>> I would guess the issue will be there too.
>>>>>
>>>>>
>>>>> Thanks a lot for any idea about what's going wrong here.
>>>>>
>>>>> Karel
>>>>> PS: resending as my former subscription to the list looks like
>>>>> stuck in
>>>>> the middle of the way. Sorry if it went through already.
>>>>
>>>> --
>>>> Tim Bish
>>>> Sr Software Engineer | RedHat Inc.
>>>> tim.bish@redhat.com | www.fusesource.com | www.redhat.com
>>>> skype: tabish121 | twitter: @tabish121
>>>> blog: http://timbish.blogspot.com/
>>>>
>>
>
>


Mime
View raw message