activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Bish <tabish...@gmail.com>
Subject Re: Reply:Reply:Re: Subscriber losing messages (from topic)
Date Fri, 11 Jan 2013 18:03:46 GMT
On Fri, 2013-01-11 at 18:29 +0100, Karel Gardas wrote: 
> Oops, sorry for the question below, now I see you've already pointed me 
> into the right direction pointing to: 
> http://activemq.apache.org/slow-consumer-handling.html
> 
> I'm going to test it.

You may also want to have a look at this Blog on the ActiveMQ
MemeroyUsage:
http://www.christianposta.com/blog/?p=273 

> 
> Thanks,
> Karel
> 
> On 01/11/13 06:26 PM, Karel Gardas wrote:
> >
> > Hi,
> >
> > thanks a lot for this fast help! Indeed, I've been using non-durable
> > subscriber so I'm glad this behavior is expected although for me it was
> > highly confusing. Thanks for clarification. Now, when I've switched to
> > using durable subscriber I see all the messages are delivered to the
> > subscriber reliably, although way much slower than in case of
> > non-durable consumer. The ratio is like few minutes versus ~170 minutes.
> > Note that I'm using ActiveMQ db on Solaris' tmp file-system so
> > effectively it's whole located in RAM.
> >
> > Now, I'm using stock ActiveMQ 5.7.0 release so I'm also using its own
> > default configuration. Ideally for our application I would expect slower
> > consumer to block faster producer in case of reaching broker internal
> > memory limit. Is there some option in ActiveMQ configuration file to
> > switch this behavior on? I've seen this was discussed on
> > http://activemq.apache.org/slow-consumers.html as an option what to do
> > in case of slower consumer so I hope it is already implemented. Also I'd
> > like to avoid durable consumers as they seems to be much slower and more
> > importantly we don't need reconnect capability and delivering of
> > messages to reconnected consumer yet.
> >
> > Thanks a lot!
> > Karel
> >
> > On 01/11/13 03:03 AM, SuoNayi wrote:
> >> Sorry for my inaccurate post.
> >> The behavior that the broker discards messages for the slow
> >> non-durable subscription
> >> depends on the maximum pending messages of the subscription.
> >> There is a pending message limit strategy for the subscription called
> >> PendingMessageLimitStrategy.
> >> You may post your broker configuration file for more details.
> >>
> >>
> >>
> >>
> >> At 2013-01-11 09:40:38,SuoNayi<suonayi2006@163.com> wrote:
> >>> Hi, I assume you're using non-durable subscription.
> >>> If so the eviction strategy will be involved when the memory
> >>> consumption of the
> >>> broker exceeds the memory limit.In your case the producer is faster
> >>> than the consumer.
> >>> This cause more and more messages are pending in memory in the broker
> >>> and eventually the broker has to use the eviction strategy to discard
> >>> messages
> >>> for the slow non-durable subscription.The default strategy is
> >>> oldestMessageEvictionStrategy
> >>> that will discard the oldest message in memory.You may take a look at
> >>> http://activemq.apache.org/slow-consumer-handling.html
> >>> In this case the behavior you observed is expected.
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> At 2013-01-11 04:59:52,"Timothy Bish"<tabish121@gmail.com> wrote:
> >>>> On Thu, 2013-01-10 at 21:52 +0100, Karel Gardas wrote:
> >>>>> Hello,
> >>>>>
> >>>>> I'm playing a bit with ActiveMQ 5.7.0 and found that under some
> >>>>> circumstances my subscriber looks like losing messages which are
> >>>>> delivered to him from the broker. Honestly speaking I don't know
if to
> >>>>> blame subscriber or broker in this case but while looking into web
> >>>>> console of the broker, the broker claims the messages are
> >>>>> delivered, but
> >>>>> I don't see them on subscriber. It took me some time to duplicate
this
> >>>>> issue on as simple as possible example, but finally I've been able
> >>>>> to a
> >>>>> bit hack activemq's own demo to duplicate it.
> >>>>>
> >>>>> If you'd like to see the same effect as I see now, just go to
> >>>>> activemq/example/src and add:
> >>>>>
> >>>>
> >>>> You should try to create a unit test to represent what's going on so
> >>>> that we can take closer look. Seems odd that the sleep would cause
> >>>> something to get lost unless there's a TTL set on the messages.
> >>>>
> >>>>> try {
> >>>>> Thread.sleep(1);
> >>>>> }
> >>>>> catch (Exception ex) {
> >>>>> ex.printStackTrace();
> >>>>> }
> >>>>>
> >>>>> into ConsumerTool.java's onMessage method. This will ensure that
> >>>>> onMessage on consumer will run a little bit slower than producer
is
> >>>>> able
> >>>>> to send messages and you will see the effect.
> >>>>>
> >>>>> Once done, just run consumer with:
> >>>>>
> >>>>> ant consumer -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
> >>>>>
> >>>>> and producer with:
> >>>>>
> >>>>> ant producer -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
> >>>>>
> >>>>> if everything is working well, then both consumer and producer should
> >>>>> end once delivering 1000000 messages. The problem is that consumer
> >>>>> does
> >>>>> not end due to losing some of the messages somewhere. If this does
not
> >>>>> happen on your box, please increase number of milliseconds on
> >>>>> Thread.sleep to make onMessage even slower. I'm running this on
> >>>>> Solaris
> >>>>> 11/JDK 1.7 on Xeon E5 2.0 GHz here.
> >>>>>
> >>>>> Now, I do have following questions:
> >>>>>
> >>>>> - am I right assuming the example above should really deliver all
the
> >>>>> messages to the consumer onMessage method?
> >>>>>
> >>>>> - is this already known issue or shall I report it properly to
> >>>>> ActiveMQ's bug tracking system?
> >>>>>
> >>>>> Just to make sure, I've also tried running consumer with ant consumer
> >>>>> -Dtopic=true -Dverbose=false -Dmax=1000000 -Dsubject=TRM
> >>>>> -Durl="tcp://localhost:61616?jms.prefetchPolicy.queuePrefetch=0"
--
> >>>>> but
> >>>>> it neither helps. At least I've thought pre-fetching might cause
> >>>>> this so
> >>>>> I tried...
> >>>>>
> >>>>> BTW: I've seen the same issue but not on demo, but on our own
> >>>>> application also while running subscriber on top of 5.6.0 and 5.5.0
> >>>>> releases. I always used 5.7.0 release for broker and producer though.
> >>>>> I've not seen it on demo as I've not tested this with older
> >>>>> releases but
> >>>>> I would guess the issue will be there too.
> >>>>>
> >>>>>
> >>>>> Thanks a lot for any idea about what's going wrong here.
> >>>>>
> >>>>> Karel
> >>>>> PS: resending as my former subscription to the list looks like
> >>>>> stuck in
> >>>>> the middle of the way. Sorry if it went through already.
> >>>>
> >>>> --
> >>>> Tim Bish
> >>>> Sr Software Engineer | RedHat Inc.
> >>>> tim.bish@redhat.com | www.fusesource.com | www.redhat.com
> >>>> skype: tabish121 | twitter: @tabish121
> >>>> blog: http://timbish.blogspot.com/
> >>>>
> >>
> >
> >
> 

-- 
Tim Bish
Sr Software Engineer | RedHat Inc.
tim.bish@redhat.com | www.fusesource.com | www.redhat.com 
skype: tabish121 | twitter: @tabish121
blog: http://timbish.blogspot.com/


Mime
View raw message