activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Posta <christian.po...@gmail.com>
Subject Re: Ignoring subscriptions to excluded destinations
Date Thu, 06 Dec 2012 21:58:38 GMT
James, this is related to a bug:
https://issues.apache.org/jira/browse/AMQ-4210
Will have a fix in shortly.


On Tue, Oct 30, 2012 at 8:52 AM, James Green <james.mk.green@gmail.com>wrote:

> Gary,
>
> Spoke config:
>         <networkConnectors>
>            <networkConnector uri="static://(tcp://
> 10.0.0.117:61616?trace=true)"
>                name="plenty-hub-01"
>                duplex="true"
>                conduitSubscriptions="false"
>                dynamicOnly="false">
>                <excludedDestinations>
>                </excludedDestinations>
>                <dynamicallyIncludedDestinations>
>                         <queue physicalName="account_events"/>
>                         <queue physicalName="accounts.ack"/>
>                         <topic physicalName="accounts"/>
>                         <topic physicalName="Account.Requests"/>
>                         <topic physicalName="Server.Requests"/>
>                </dynamicallyIncludedDestinations>
>            </networkConnector>
>        </networkConnectors>
>
> On the hub, I captured this: http://pastebin.com/xVd2d0jw
>
> Hope this is adequate to be getting on with. We have got our scenario
> working, but only with both dynamicallyIncludedDestinations and a
> destinationsFilter set.
>
> Let me know if this doesn't cover enough!
>
> James
>
> On 30 October 2012 15:03, Gary Tully <gary.tully@gmail.com> wrote:
>
> > that does seem odd, enable trace=true on the tcp transport to see the
> > advisory consumer subscription command from the bridge, and the
> > corresponding advisory composite destination. This will all be part of
> > bridge setup and should be informative.
> >
> > you need to enable debug logging for
> > org.apache.activemq.transport.TransportLogger to see it.
> >
> > <networkConnector uri="static://(tcp://10.0.0.117:61616?trace=true)"
> >
> > On 30 October 2012 14:39, James Green <james.mk.green@gmail.com> wrote:
> > > Gary,
> > >
> > > On our test cluster one of our spokes has been configured thus:
> > >
> > >         <networkConnectors>
> > >            <networkConnector uri="static://(tcp://10.0.0.117:61616)"
> > >                name="plenty-hub-01"
> > >                duplex="true"
> > >                conduitSubscriptions="false"
> > >                dynamicOnly="false">
> > >                <excludedDestinations>
> > >                </excludedDestinations>
> > >                <dynamicallyIncludedDestinations>
> > >                         <queue physicalName="account_events"/>
> > >                         <queue physicalName="accounts.ack"/>
> > >                </dynamicallyIncludedDestinations>
> > >            </networkConnector>
> > >        </networkConnectors>
> > >
> > > .117 is our test hub.
> > >
> > > Against the spoke we subscribed to the Outbound.Account.200002 queue,
> and
> > > .117 logged the consumer and that it had to ignore it.
> > >
> > > We even restarted the hub's AMQ instance and deleted the advisory topic
> > on
> > > the hub that had been created. Still, we get traffic.
> > >
> > > I am now at a loss to explain this traffic. Is there something else we
> > must
> > > do?
> > >
> > > Thanks,
> > >
> > > James
> > >
> > > On 30 October 2012 14:22, Gary Tully <gary.tully@gmail.com> wrote:
> > >
> > >> there are two different filters involved.
> > >>
> > >> The bridge gets notification of demand on a destination, ie:
> > >> consumers, using an advisory consumer.
> > >> To reduce the advisory traffic, the set of matching advisory
> > >> destinations subscribed to can be filtered. Using the
> > >> destinationFilter property or generated from the dynamically included
> > >> list.
> > >>
> > >> If the advisory consumer does not listen to specific destinations then
> > >> it won't ever get notification of them and won't ever have to filter
> > >> at the bridge level.
> > >>
> > >> The list of exclusions only works at the bridge level, where it
> > >> processes advisory messages, so is independent of the
> > >> destinationFilter.
> > >>
> > >> So for your whitelist, just deal with application destinations, the
> > >> corresponding advisory destinations will be subscribed to
> > >> automatically.
> > >>
> > >>
> > >> On 30 October 2012 12:40, James Green <james.mk.green@gmail.com>
> wrote:
> > >> > http://activemq.apache.org/networks-of-brokers.html states for
> > >> > dynamicallyIncludedDestinations that "destinations that match this
> > list *
> > >> > will* be forwarded across the network *n.b.* an empty list means all
> > >> > destinations not in the exluded list will be forwarded".
> > >> >
> > >> > This reads to me that, given an empty dynamic list and a set of
> > >> exclusions,
> > >> > only those exclusions will be totally filtered. That's not the case
> at
> > >> all
> > >> > (evidently).
> > >> >
> > >> > We will try to build a list of dynamic inclusions (a whitelist
> instead
> > >> of a
> > >> > blacklist). Do we need to include topics for advisories on the
> queues
> > or
> > >> > just the queues themselves? Otherwise presumably the consumer
> > >> subscription
> > >> > information on queues/topics sent from hub to spoke won't get
> notices
> > on
> > >> > the hub and thus the hub will not send the messages to the hub on
> > demand?
> > >> >
> > >> > Thanks,
> > >> > James
> > >> >
> > >> > On 30 October 2012 12:32, Gary Tully <gary.tully@gmail.com>
wrote:
> > >> >
> > >> >> in your case you need to provide a destinationFilter b/c it is
only
> > >> >> auto built from <dynamicalyIncludedDestinations> not excluded
> > >> >> destinations.
> > >> >>
> > >> >> Do you know up front what you want to bridge or just what you
want
> to
> > >> >> ignore?
> > >> >>
> > >> >> A the moment, afaik,  we don't have a way of providing a negative
> > >> >> filter which seems to be what you want.
> > >> >>
> > >> >> On 30 October 2012 12:08, James Green <james.mk.green@gmail.com>
> > wrote:
> > >> >> > Gary,
> > >> >> >
> > >> >> > I think you're saying that subscription advisories for excluded
> > >> >> > destinations should be suppressed.
> > >> >> >
> > >> >> > On the hub we're seeing advisories for queues on that spoke.
Is
> > there
> > >> >> > therefore a bug?
> > >> >> >
> > >> >> > James
> > >> >> >
> > >> >> > On 30 October 2012 11:58, Gary Tully <gary.tully@gmail.com>
> wrote:
> > >> >> >
> > >> >> >> the destinationFilter does the job of narrowing the list
of
> > >> >> >> interesting consumers by limiting the advisory consumer
to a
> > subset
> > >> of
> > >> >> >> destinations.
> > >> >> >> This is auto generated if it is not configured from 5.6.0,
but
> > needs
> > >> >> >> both ends of the networkconnector to be => 5.6
> > >> >> >>
> > >> >> >> Have a peek at:
> > >> >> >> https://issues.apache.org/jira/browse/AMQ-3384
> > >> >> >>
> > >> >> >>
> > >> >> >> On 30 October 2012 09:10, James Green <james.mk.green@gmail.com
> >
> > >> wrote:
> > >> >> >> > Part of my intention of declaring excluded destinations
was to
> > >> reduce
> > >> >> the
> > >> >> >> > amount of traffic over the ADSL line that exists
between hub
> and
> > >> the
> > >> >> >> spokes.
> > >> >> >> >
> > >> >> >> > However, despite the instruction to ban messaging
on these
> > >> >> destinations,
> > >> >> >> > the amount of traffic and instructions that the
hub receives
> has
> > >> not
> > >> >> >> > changed.
> > >> >> >> >
> > >> >> >> > The documentation, in my opinion, gives the impression
that
> > >> excluded
> > >> >> >> > destinations is to segregate the network; actually
it only
> > >> performs a
> > >> >> >> very
> > >> >> >> > thin segregation. People wanting to reduce the bandwidth
> between
> > >> nodes
> > >> >> >> will
> > >> >> >> > use this and may be rather disappointed by the results...
> > >> >> >> >
> > >> >> >> > James
> > >> >> >> >
> > >> >> >> > On 29 October 2012 22:23, Christian Posta <
> > >> christian.posta@gmail.com>
> > >> >> >> wrote:
> > >> >> >> >
> > >> >> >> >> >
> > >> >> >> >> > I was expecting to see no traffic of any
kind on our hub
> > >> concerning
> > >> >> >> >> > Outbound.Account.>, yet sub requests
are still flooding in.
> > >> >> >> >>
> > >> >> >> >> Can you explain more what you mean? Do you see
subs being
> > created
> > >> for
> > >> >> >> that
> > >> >> >> >> dest on the networked brokers?
> > >> >> >> >>
> > >> >> >> >> If what you mean is you're seeing the logs below,
that's as
> > >> intended.
> > >> >> >> When
> > >> >> >> >> a bridge is established, it will listen to the
remote
> broker's
> > >> >> consumer
> > >> >> >> >> advisory messages (it listens to all of them,
they are not
> > >> filtered).
> > >> >> >> If it
> > >> >> >> >> sees a consumerInfo come in for a destination
that is
> > excluded, it
> > >> >> will
> > >> >> >> >> just ignore it and log the message you see below.
This is by
> > >> design,
> > >> >> at
> > >> >> >> the
> > >> >> >> >> moment.
> > >> >> >> >>
> > >> >> >> >> On Mon, Oct 29, 2012 at 5:59 AM, James Green
<
> > >> >> james.mk.green@gmail.com
> > >> >> >> >> >wrote:
> > >> >> >> >>
> > >> >> >> >> > Given:
> > >> >> >> >> >
> > >> >> >> >> >         <networkConnectors>
> > >> >> >> >> >             <networkConnector
> > uri="static://(ssl://hub:61617)"
> > >> >> >> >> >                 name="hub"
> > >> >> >> >> >                 duplex="true"
> > >> >> >> >> >                 conduitSubscriptions="false"
> > >> >> >> >> >                 dynamicOnly="false">
> > >> >> >> >> >                 <excludedDestinations>
> > >> >> >> >> >                     <queue
> > physicalName="Outbound.Account.>"/>
> > >> >> >> >> >                 </excludedDestinations>
> > >> >> >> >> >                 <staticallyIncludedDestinations>
> > >> >> >> >> >                 </staticallyIncludedDestinations>
> > >> >> >> >> >             </networkConnector>
> > >> >> >> >> >         </networkConnectors>
> > >> >> >> >> >
> > >> >> >> >> > On "hub" I see:
> > >> >> >> >> >
> > >> >> >> >> > 2012-10-29 12:44:34,722 | DEBUG | hub Ignoring
sub from
> > zorin,
> > >> >> >> >> destination
> > >> >> >> >> > queue://Outbound.Account.20481 is not permiited
> :ConsumerInfo
> > >> >> >> {commandId
> > >> >> >> >> =
> > >> >> >> >> > 5, responseRequired = false, consumerId
=
> > >> >> >> >> > ID:quarrel-40451-1351260922652-4:760216:-1:2,
destination =
> > >> >> >> >> > queue://Outbound.Account.20481, prefetchSize
= 1,
> > >> >> >> >> > maximumPendingMessageLimit = 0, browser
= false,
> > dispatchAsync =
> > >> >> true,
> > >> >> >> >> > selector = MJStage = 'Dispatch', subscriptionName
= null,
> > >> noLocal =
> > >> >> >> >> false,
> > >> >> >> >> > exclusive = false, retroactive = false,
priority = 0,
> > >> brokerPath =
> > >> >> >> null,
> > >> >> >> >> > optimizedAcknowledge = false, noRangeAcks
= false,
> > >> >> >> additionalPredicate =
> > >> >> >> >> > null} |
> > >> org.apache.activemq.network.DemandForwardingBridgeSupport |
> > >> >> >> >> > ActiveMQ Transport: ssl:///n.n.n.n:32831
> > >> >> >> >> >
> > >> >> >> >> > I was expecting to see no traffic of any
kind on our hub
> > >> concerning
> > >> >> >> >> > Outbound.Account.>, yet sub requests
are still flooding in.
> > >> >> >> >> >
> > >> >> >> >> > Is this normal? Can I get my desired result?
> > >> >> >> >> >
> > >> >> >> >> > Thanks,
> > >> >> >> >> >
> > >> >> >> >> > James
> > >> >> >> >> >
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >>
> > >> >> >> >> --
> > >> >> >> >> *Christian Posta*
> > >> >> >> >> http://www.christianposta.com/blog
> > >> >> >> >> twitter: @christianposta
> > >> >> >> >>
> > >> >> >>
> > >> >> >>
> > >> >> >>
> > >> >> >> --
> > >> >> >> http://redhat.com
> > >> >> >> http://blog.garytully.com
> > >> >> >>
> > >> >>
> > >> >>
> > >> >>
> > >> >> --
> > >> >> http://redhat.com
> > >> >> http://blog.garytully.com
> > >> >>
> > >>
> > >>
> > >>
> > >> --
> > >> http://redhat.com
> > >> http://blog.garytully.com
> > >>
> >
> >
> >
> > --
> > http://redhat.com
> > http://blog.garytully.com
> >
>



-- 
*Christian Posta*
http://www.christianposta.com/blog
twitter: @christianposta

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message