activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino" <hi...@hiramchirino.com>
Subject Re: handling slow consumers on non-persistent topics and AMQ-688
Date Tue, 04 Jul 2006 13:21:32 GMT
The problem is there is a high likelyhood that we will need to also extend
the MessageStore interface to provide some cursoring ablilities.  So the
change is not going to just be sefcontained in the broker region package.

On 7/4/06, James Strachan <james.strachan@gmail.com> wrote:
>
> Are you totally convinced it requires a new branch; e.g. what if we
> created a whole new Region implementation in another package or
> something?
>
> On 7/4/06, Hiram Chirino <hiram@hiramchirino.com> wrote:
> > I think that fixing these issues are going to require substancial work
> on
> > internals of the broker.  I want to try to take a stab fixing this stuff
> but
> > since it's going to take a few iterations to get right, how about we
> branch
> > trunk and work there to avoid breaking everbody else?  Any sugestions
> for
> > the branch name?
> >
> > Regards,
> > Hiram
> >
> > On 7/3/06, James Strachan <james.strachan@gmail.com> wrote:
> > >
> > > While perusing JIRA I spotted this issue again...
> > > http://issues.apache.org/activemq/browse/AMQ-688
> > > I know its an issue close to folks at Amazon's hearts.
> > >
> > > Dealing with slow consumers is a fascinating problem for a messaging
> > > system; its quite a tricky problem :). Here's some background on the
> > > issue...
> > > http://incubator.apache.org/activemq/slow-consumers.html
> > >
> > > together with the currently supported features - to allow messages to
> > > be discarded on slow consumers using a pluggable algorithm...
> > > http://incubator.apache.org/activemq/slow-consumer-handling.html
> > >
> > > Now for all consumers we fill up prefetch buffers as quickly as
> > > possible...
> > >
> http://incubator.apache.org/activemq/what-is-the-prefetch-limit-for.html
> > >
> > > so there's always a buffer of messages per consumer. For non-durable
> > > topics once these messages are written to a socket they are evicted
> > > from RAM; so we already have some support for slow consumers.
> > >
> > > I wanted to start a discussion on both AMQ-688 and to see if folks had
> > > other requirements for handling slow consumers & to try decide what
> > > features & stragegies we should add next in this area.
> > >
> > > One of the first requirements folks ask for is that rather than
> > > blocking permanently the non-persistent topic engine until RAM is
> > > cleared, that at a certain threshhold we start spooling to disk. I've
> > > raised a separate JIRA issue for this specific feature request...
> > >
> > > http://issues.apache.org/activemq/browse/AMQ-791
> > >
> > > Another issue some folks have hit in the past is that for high
> > > performance and to minimise context switches in the broker, we tend to
> > > use the current thread in the broker to dispatch to all the
> > > non-durable topic consumers so a slow/blocked consumer can appear to
> > > 'block' the producer.
> > >
> > > I've raised this issue to track that feature
> > > http://issues.apache.org/activemq/browse/AMQ-792
> > >
> > > I just wondered if folks had any other issues or requirements with the
> > > whole slow consumers and non-durable topics they'd like to discuss? Is
> > > there any requirements we won't have covered if the above two JIRAs
> > > are fixed
> > > --
> > >
> > > James
> > > -------
> > > http://radio.weblogs.com/0112098/
> > >
> >
> >
> >
> > --
> > Regards,
> > Hiram
> >
> > Blog: http://hiramchirino.com
> >
> >
>
>
> --
>
> James
> -------
> http://radio.weblogs.com/0112098/
>



-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message