activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Green <james.mk.gr...@gmail.com>
Subject Re: Ignoring subscriptions to excluded destinations
Date Tue, 30 Oct 2012 15:52:43 GMT
Gary,

Spoke config:
        <networkConnectors>
           <networkConnector uri="static://(tcp://
10.0.0.117:61616?trace=true)"
               name="plenty-hub-01"
               duplex="true"
               conduitSubscriptions="false"
               dynamicOnly="false">
               <excludedDestinations>
               </excludedDestinations>
               <dynamicallyIncludedDestinations>
                        <queue physicalName="account_events"/>
                        <queue physicalName="accounts.ack"/>
                        <topic physicalName="accounts"/>
                        <topic physicalName="Account.Requests"/>
                        <topic physicalName="Server.Requests"/>
               </dynamicallyIncludedDestinations>
           </networkConnector>
       </networkConnectors>

On the hub, I captured this: http://pastebin.com/xVd2d0jw

Hope this is adequate to be getting on with. We have got our scenario
working, but only with both dynamicallyIncludedDestinations and a
destinationsFilter set.

Let me know if this doesn't cover enough!

James

On 30 October 2012 15:03, Gary Tully <gary.tully@gmail.com> wrote:

> that does seem odd, enable trace=true on the tcp transport to see the
> advisory consumer subscription command from the bridge, and the
> corresponding advisory composite destination. This will all be part of
> bridge setup and should be informative.
>
> you need to enable debug logging for
> org.apache.activemq.transport.TransportLogger to see it.
>
> <networkConnector uri="static://(tcp://10.0.0.117:61616?trace=true)"
>
> On 30 October 2012 14:39, James Green <james.mk.green@gmail.com> wrote:
> > Gary,
> >
> > On our test cluster one of our spokes has been configured thus:
> >
> >         <networkConnectors>
> >            <networkConnector uri="static://(tcp://10.0.0.117:61616)"
> >                name="plenty-hub-01"
> >                duplex="true"
> >                conduitSubscriptions="false"
> >                dynamicOnly="false">
> >                <excludedDestinations>
> >                </excludedDestinations>
> >                <dynamicallyIncludedDestinations>
> >                         <queue physicalName="account_events"/>
> >                         <queue physicalName="accounts.ack"/>
> >                </dynamicallyIncludedDestinations>
> >            </networkConnector>
> >        </networkConnectors>
> >
> > .117 is our test hub.
> >
> > Against the spoke we subscribed to the Outbound.Account.200002 queue, and
> > .117 logged the consumer and that it had to ignore it.
> >
> > We even restarted the hub's AMQ instance and deleted the advisory topic
> on
> > the hub that had been created. Still, we get traffic.
> >
> > I am now at a loss to explain this traffic. Is there something else we
> must
> > do?
> >
> > Thanks,
> >
> > James
> >
> > On 30 October 2012 14:22, Gary Tully <gary.tully@gmail.com> wrote:
> >
> >> there are two different filters involved.
> >>
> >> The bridge gets notification of demand on a destination, ie:
> >> consumers, using an advisory consumer.
> >> To reduce the advisory traffic, the set of matching advisory
> >> destinations subscribed to can be filtered. Using the
> >> destinationFilter property or generated from the dynamically included
> >> list.
> >>
> >> If the advisory consumer does not listen to specific destinations then
> >> it won't ever get notification of them and won't ever have to filter
> >> at the bridge level.
> >>
> >> The list of exclusions only works at the bridge level, where it
> >> processes advisory messages, so is independent of the
> >> destinationFilter.
> >>
> >> So for your whitelist, just deal with application destinations, the
> >> corresponding advisory destinations will be subscribed to
> >> automatically.
> >>
> >>
> >> On 30 October 2012 12:40, James Green <james.mk.green@gmail.com> wrote:
> >> > http://activemq.apache.org/networks-of-brokers.html states for
> >> > dynamicallyIncludedDestinations that "destinations that match this
> list *
> >> > will* be forwarded across the network *n.b.* an empty list means all
> >> > destinations not in the exluded list will be forwarded".
> >> >
> >> > This reads to me that, given an empty dynamic list and a set of
> >> exclusions,
> >> > only those exclusions will be totally filtered. That's not the case at
> >> all
> >> > (evidently).
> >> >
> >> > We will try to build a list of dynamic inclusions (a whitelist instead
> >> of a
> >> > blacklist). Do we need to include topics for advisories on the queues
> or
> >> > just the queues themselves? Otherwise presumably the consumer
> >> subscription
> >> > information on queues/topics sent from hub to spoke won't get notices
> on
> >> > the hub and thus the hub will not send the messages to the hub on
> demand?
> >> >
> >> > Thanks,
> >> > James
> >> >
> >> > On 30 October 2012 12:32, Gary Tully <gary.tully@gmail.com> wrote:
> >> >
> >> >> in your case you need to provide a destinationFilter b/c it is only
> >> >> auto built from <dynamicalyIncludedDestinations> not excluded
> >> >> destinations.
> >> >>
> >> >> Do you know up front what you want to bridge or just what you want
to
> >> >> ignore?
> >> >>
> >> >> A the moment, afaik,  we don't have a way of providing a negative
> >> >> filter which seems to be what you want.
> >> >>
> >> >> On 30 October 2012 12:08, James Green <james.mk.green@gmail.com>
> wrote:
> >> >> > Gary,
> >> >> >
> >> >> > I think you're saying that subscription advisories for excluded
> >> >> > destinations should be suppressed.
> >> >> >
> >> >> > On the hub we're seeing advisories for queues on that spoke. Is
> there
> >> >> > therefore a bug?
> >> >> >
> >> >> > James
> >> >> >
> >> >> > On 30 October 2012 11:58, Gary Tully <gary.tully@gmail.com>
wrote:
> >> >> >
> >> >> >> the destinationFilter does the job of narrowing the list of
> >> >> >> interesting consumers by limiting the advisory consumer to
a
> subset
> >> of
> >> >> >> destinations.
> >> >> >> This is auto generated if it is not configured from 5.6.0,
but
> needs
> >> >> >> both ends of the networkconnector to be => 5.6
> >> >> >>
> >> >> >> Have a peek at:
> >> >> >> https://issues.apache.org/jira/browse/AMQ-3384
> >> >> >>
> >> >> >>
> >> >> >> On 30 October 2012 09:10, James Green <james.mk.green@gmail.com>
> >> wrote:
> >> >> >> > Part of my intention of declaring excluded destinations
was to
> >> reduce
> >> >> the
> >> >> >> > amount of traffic over the ADSL line that exists between
hub and
> >> the
> >> >> >> spokes.
> >> >> >> >
> >> >> >> > However, despite the instruction to ban messaging on
these
> >> >> destinations,
> >> >> >> > the amount of traffic and instructions that the hub receives
has
> >> not
> >> >> >> > changed.
> >> >> >> >
> >> >> >> > The documentation, in my opinion, gives the impression
that
> >> excluded
> >> >> >> > destinations is to segregate the network; actually it
only
> >> performs a
> >> >> >> very
> >> >> >> > thin segregation. People wanting to reduce the bandwidth
between
> >> nodes
> >> >> >> will
> >> >> >> > use this and may be rather disappointed by the results...
> >> >> >> >
> >> >> >> > James
> >> >> >> >
> >> >> >> > On 29 October 2012 22:23, Christian Posta <
> >> christian.posta@gmail.com>
> >> >> >> wrote:
> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > I was expecting to see no traffic of any kind
on our hub
> >> concerning
> >> >> >> >> > Outbound.Account.>, yet sub requests are
still flooding in.
> >> >> >> >>
> >> >> >> >> Can you explain more what you mean? Do you see subs
being
> created
> >> for
> >> >> >> that
> >> >> >> >> dest on the networked brokers?
> >> >> >> >>
> >> >> >> >> If what you mean is you're seeing the logs below,
that's as
> >> intended.
> >> >> >> When
> >> >> >> >> a bridge is established, it will listen to the remote
broker's
> >> >> consumer
> >> >> >> >> advisory messages (it listens to all of them, they
are not
> >> filtered).
> >> >> >> If it
> >> >> >> >> sees a consumerInfo come in for a destination that
is
> excluded, it
> >> >> will
> >> >> >> >> just ignore it and log the message you see below.
This is by
> >> design,
> >> >> at
> >> >> >> the
> >> >> >> >> moment.
> >> >> >> >>
> >> >> >> >> On Mon, Oct 29, 2012 at 5:59 AM, James Green <
> >> >> james.mk.green@gmail.com
> >> >> >> >> >wrote:
> >> >> >> >>
> >> >> >> >> > Given:
> >> >> >> >> >
> >> >> >> >> >         <networkConnectors>
> >> >> >> >> >             <networkConnector
> uri="static://(ssl://hub:61617)"
> >> >> >> >> >                 name="hub"
> >> >> >> >> >                 duplex="true"
> >> >> >> >> >                 conduitSubscriptions="false"
> >> >> >> >> >                 dynamicOnly="false">
> >> >> >> >> >                 <excludedDestinations>
> >> >> >> >> >                     <queue
> physicalName="Outbound.Account.>"/>
> >> >> >> >> >                 </excludedDestinations>
> >> >> >> >> >                 <staticallyIncludedDestinations>
> >> >> >> >> >                 </staticallyIncludedDestinations>
> >> >> >> >> >             </networkConnector>
> >> >> >> >> >         </networkConnectors>
> >> >> >> >> >
> >> >> >> >> > On "hub" I see:
> >> >> >> >> >
> >> >> >> >> > 2012-10-29 12:44:34,722 | DEBUG | hub Ignoring
sub from
> zorin,
> >> >> >> >> destination
> >> >> >> >> > queue://Outbound.Account.20481 is not permiited
:ConsumerInfo
> >> >> >> {commandId
> >> >> >> >> =
> >> >> >> >> > 5, responseRequired = false, consumerId =
> >> >> >> >> > ID:quarrel-40451-1351260922652-4:760216:-1:2,
destination =
> >> >> >> >> > queue://Outbound.Account.20481, prefetchSize
= 1,
> >> >> >> >> > maximumPendingMessageLimit = 0, browser = false,
> dispatchAsync =
> >> >> true,
> >> >> >> >> > selector = MJStage = 'Dispatch', subscriptionName
= null,
> >> noLocal =
> >> >> >> >> false,
> >> >> >> >> > exclusive = false, retroactive = false, priority
= 0,
> >> brokerPath =
> >> >> >> null,
> >> >> >> >> > optimizedAcknowledge = false, noRangeAcks =
false,
> >> >> >> additionalPredicate =
> >> >> >> >> > null} |
> >> org.apache.activemq.network.DemandForwardingBridgeSupport |
> >> >> >> >> > ActiveMQ Transport: ssl:///n.n.n.n:32831
> >> >> >> >> >
> >> >> >> >> > I was expecting to see no traffic of any kind
on our hub
> >> concerning
> >> >> >> >> > Outbound.Account.>, yet sub requests are
still flooding in.
> >> >> >> >> >
> >> >> >> >> > Is this normal? Can I get my desired result?
> >> >> >> >> >
> >> >> >> >> > Thanks,
> >> >> >> >> >
> >> >> >> >> > James
> >> >> >> >> >
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> *Christian Posta*
> >> >> >> >> http://www.christianposta.com/blog
> >> >> >> >> twitter: @christianposta
> >> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> http://redhat.com
> >> >> >> http://blog.garytully.com
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> http://redhat.com
> >> >> http://blog.garytully.com
> >> >>
> >>
> >>
> >>
> >> --
> >> http://redhat.com
> >> http://blog.garytully.com
> >>
>
>
>
> --
> http://redhat.com
> http://blog.garytully.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message