activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: prefetchExtension off-by-1 for transacted consumers with prefetchSize > 0
Date Mon, 22 Mar 2010 15:37:40 GMT
If you are in the thick of it and can propose a patch that would be great.
A new boolean option on ActiveMQPolicy would be the best way to introduce
the configuration on a destination basis (set in
org.apache.activemq.broker.region.policy.PolicyEntry.configure(Broker,
SystemUsage, QueueSubscription) and if you want more fine grain control, an
option can be added to ConsumerInfo to allow the configuration on a per
subscription basis, propagating the change though to a connection and
connection factory. That will require regenerating the wireformat
marshallers (in activemq-core; mvn -Popenwire-generate generate-sources) so
that the new option is propagated from the client. See how you get on.

Otherwise, we can try and get this change into the 5.4 release.

BTW: Is there a problem with the activemq RA that stops you from using it?


On 22 March 2010 14:56, rodos77 <eugene.rodos@nexj.com> wrote:

>
> I would point out that another JIRA that I've opened (
> https://issues.apache.org/activemq/browse/AMQ-2652 AMQ-2652 ), illustrates
> a
> deadlock (hang) which is triggered precisely by the prefetch extension of
> transacted consumers (see JIRA for details).  So while trying to avoid one
> potential hang, the use of prefetch extension causes another.
>
> However, the solution that you propose to make the prefetch extension
> configurable and to have it off by default sounds good to me.  How should
> we
> proceed to have this put in place?
>
>
>
> Gary Tully wrote:
> >
> > Fair point. From the perspective of the ResourceAdapter where there are
> > explicit expectations this makes good sense. The prefetch should be
> > written
> > in stone.
> >
> > From a regular transacted consumers perspective, where the prefetch is
> > largely hidden, avoiding a hang for large transactions is good because a
> > hang would be hard to diagnose. I agree it could be considered user/error
> > but it is difficult to flag it as such.
> >
> > It may make sense to make the (extend the prefetch for a
> > transaction) behavior configurable such that it can be disabled for the
> RA
> > or off by default and enabled in the event that large transactions hang.
> > Thinking more, off by default is probably the best approach as
> > transactions
> > that exceed the prefetch should be rare and the RAR behavior would match
> > expectations.
> >
> > On 18 March 2010 22:05, rodos77 <eugene.rodos@nexj.com> wrote:
> >
> >>
> >> I don't really get this.  If you extend the prefetchSize to more than
> >> what
> >> it
> >> was configured to be by the maxMessages parameter in the
> >> Connection.createConnectionConsumer() call, you are violating the JMS
> >> spec.
> >> The maxMessages parameter is defined as "the maximum number of messages
> >> that
> >> can be assigned to a server session at one time", so how can you make it
> >> bigger than what it was set to by the ConnectionConsumer?!
> >>
> >> ActiveMQ Resource Adapter has 2 parameters in its ActivationSpec that
> >> control the maxMessages argument used in the
> >> Connection.createConnectionConsumer() call: maxSessions and
> >> maxMessagesPerSessions.  The maxMessages argument is set to their
> >> product.
> >> This makes sense.  If a user configures ActiveMQ to have x maxSessions
> >> and
> >> y
> >> maxMessagesPerSessions, maxMessages (and consequentially the
> >> prefetchSize)
> >> should be set to x*y.  But why should this ever be extended?  Shouldn't
> >> it
> >> be expected that only a maximum of x*y messages be delivered at the same
> >> time?  If a user wants to process z <= x*y messages in the same tx, that
> >> should not pose any problem as the RA will have x ServerSessions each
> >> capable of processing y messages at the same time.  But if the user
> wants
> >> to
> >> process z > x*y messages at the same time after explicitly configuring
> >> ActiveMQ to only deliver a maximum of x*y messages at the same time,
> this
> >> is
> >> a user error/problem.  ActiveMQ should not be violating the JMS Spec to
> >> accommodate this.  What is the point of having those properties in the
> >> 1st
> >> place if you are going to extend them and deliver as many messages as
> >> available?
> >>
> >>
> >>
> >> Gary Tully wrote:
> >> >
> >> > The prefetch extension in the transacted case is necessary for the
> case
> >> > where a transacted consumer wants to consumer more than prefetch
> >> messages
> >> > in
> >> > a single transaction. Without the extension, the broker would see a
> >> full
> >> > prefetch and not dispatch any more and a receive could block.
> >> >
> >> > On 15 March 2010 21:31, rodos77 <eugene.rodos@nexj.com> wrote:
> >> >
> >> >>
> >> >> JIRA AMQ-2651 opened.
> >> >>
> >> >> I would be happy to provide a patch but as I stated in my original
> >> post,
> >> >> I
> >> >> don't fully understand the need for prefetchExtension in the case of
> a
> >> >> transacted, asynchronous (prefetchSize > 0) consumer.  If somebody
> >> could
> >> >> explain it, that would perhaps give me the required knowledge to come
> >> up
> >> >> with a patch.
> >> >>
> >> >> The way I see it now, I would get rid of the prefetchExtension (in
> the
> >> >> above
> >> >> mentioned case) all together, but again I'm probably missing some
> >> piece
> >> >> of
> >> >> information and I don't want to cause any side effects.  Barring
> that,
> >> I
> >> >> think there is an off-by-1 error in its calculation but I need this
> >> >> confirmed in order to create a proper patch.
> >> >> --
> >> >> View this message in context:
> >> >>
> >>
> http://old.nabble.com/prefetchExtension-off-by-1-for-transacted-consumers-with-prefetchSize-%3E-0-tp27866123p27910673.html
> >> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >> >>
> >> >>
> >> >
> >> >
> >> > --
> >> > http://blog.garytully.com
> >> >
> >> > Open Source Integration
> >> > http://fusesource.com
> >> >
> >> >
> >>
> >> --
> >> View this message in context:
> >>
> http://old.nabble.com/prefetchExtension-off-by-1-for-transacted-consumers-with-prefetchSize-%3E-0-tp27866123p27950855.html
> >> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
> >>
> >>
> >
> >
> > --
> > http://blog.garytully.com
> >
> > Open Source Integration
> > http://fusesource.com
> >
> >
>
> --
> View this message in context:
> http://old.nabble.com/prefetchExtension-off-by-1-for-transacted-consumers-with-prefetchSize-%3E-0-tp27866123p27987505.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>


-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message