activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martyn Taylor <mtay...@redhat.com>
Subject Re: ActiveMQConnectionFactory: use initial connectors instead of received topology
Date Fri, 04 Aug 2017 09:07:30 GMT
Just catching up on this.  Firstly I don't think this is a bug.  The static
connector list is used for discovery, not for defining a set of brokers to
round robin over, I agree it's a limitation when used in conjunction with
the out of the box round robin policy.  However, it works just fine with
UDP.

The second point I have is that load balancing policy is plugable, if there
are specific use cases that require different behaviour, the the default
out of the box policies you can create your own policy or modify the
default round robin.

The out of the box round robin policy is a bit basic and only evenly
distributes load on a per client basis.  It doesn't help if there are lots
of clients with one connection.  A better approach imo would be to
implement a LowestNumberOfConnectionsPolicy, where the broker sends the
number of connections per node in the topology (maybe it's already there, I
need to check).  The client can use this metric to decide where to connect
next.  This only works with the CORE protocol of course.  To do this server
side, we could use redirects (I realise not currently implemented in some
protocols).

The small change Michael suggested does look to solve the problem for the
basic out of the box policy.

Cheers

On Thu, Aug 3, 2017 at 8:07 PM, Justin Bertram <jbertram@redhat.com> wrote:

> That looks like a good approach to me.
>
>
> Justin
>
> On Thu, Aug 3, 2017 at 2:02 PM, Michael André Pearce <
> michael.andre.pearce@me.com> wrote:
>
> > That's fair enough.
> >
> > Going with then the other option of skipping the node if it's next in the
> > list. That works if the node comes back in the first position but you'll
> > still end up with the same problem if it is in the second position.
> >
> > Eg initial node order:
> >
> > A B C
> >
> > Topology
> >
> > B A C
> >
> > First three connections would be
> >
> > A B A
> >
> > in that case.
> >
> > Could I suggest maybe enhance it a little so the node that made the
> > initial connection is moved to the end of the list/array in the topology.
> > As such you get a truer round robin.
> >
> > Eg initial static list:
> > A B C
> >
> > Connection made on A.
> >
> > Topology returned:
> >
> > A B C
> >
> > Reorder it moving A to the end.
> >
> > B C A
> >
> > As such first three connections will be:
> >
> > A B C
> >
> > And after continue to be evenly round robin'd
> >
> > This way it's a small reorder, get a balanced number of connections and
> > you don't waste that first connection.
> >
> > WDYT?
> >
> > Sent from my iPhone
> >
> > > On 3 Aug 2017, at 19:02, Justin Bertram <jbertram@redhat.com> wrote:
> > >
> > > As stated previously, I'm not in favor of using the initial connection
> > only
> > > for discovery as I think it's wasteful.
> > >
> > >
> > > Justin
> > >
> > > On Thu, Aug 3, 2017 at 12:41 PM, Michael André Pearce <
> > > michael.andre.pearce@me.com> wrote:
> > >
> > >> How much would it be for it to also only use the topology eg the
> > solution
> > >> that just uses the first connection for discovery only, and then
> > topology
> > >> for the actual connections, as so then at least support expanding the
> > >> cluster also but also load balancing better the initial connections.
> > That
> > >> way can support all three options.
> > >>
> > >> Sent from my iPhone
> > >>
> > >>> On 3 Aug 2017, at 18:24, Justin Bertram <jbertram@redhat.com>
wrote:
> > >>>
> > >>> I see your point.  I supposed I was focusing on the title of his
> > message
> > >>> (i.e. "use initial connectors instead of received topology") rather
> > than
> > >>> the the details in the description of his problem.
> > >>>
> > >>> FWIW, I plan on implementing a new connector parameter to support
> > >> ignoring
> > >>> the topology because I think it will be useful for this use-case as
> > well
> > >> as
> > >>> some others I've run across.
> > >>>
> > >>>
> > >>> Justin
> > >>>
> > >>> On Thu, Aug 3, 2017 at 11:45 AM, Michael André Pearce <
> > >>> michael.andre.pearce@me.com> wrote:
> > >>>
> > >>>> From what I read the double connection due to using both lists
is
> the
> > >>>> underlying issue from the original mail.
> > >>>>
> > >>>> "
> > >>>> There is a number of problems and inneficiencies we see on this
> > >> approach.
> > >>>> If we have a cluster with 3 hosts for example, and we declare those
> on
> > >> the
> > >>>> host list and get 3 connections using the round robin policy, we
> would
> > >>>> expect to get one connection for each host. But that's not what
> > happens.
> > >>>> The
> > >>>> load balancing policy starts iterating over one list (the initial
> > >> connector
> > >>>> list) and after the first successfull connection it continues
> > iterating
> > >>>> over
> > >>>> another list (the received topology), so most of the time you would
> > get
> > >> two
> > >>>> connections to the same host and none for one of them.
> > >>>> "
> > >>>>
> > >>>> Sent from my iPhone
> > >>>>
> > >>>>> On 3 Aug 2017, at 17:41, Justin Bertram <jbertram@redhat.com>
> wrote:
> > >>>>>
> > >>>>> IMO the way to deal with this is to just add a bit of logic
so you
> > >> don't
> > >>>>> get 2 consecutive connections to the same host.  Making a
> connection
> > >> with
> > >>>>> the static connectors, getting the topology, closing the
> connection,
> > >> and
> > >>>>> then making another connection with the topology is wasteful.
> > >>>>>
> > >>>>> In any case, an option not to use the topology at all would
be
> > useful.
> > >>>>> That is, after all, what this thread is really about.
> > >>>>>
> > >>>>>
> > >>>>> Justin
> > >>>>>
> > >>>>> On Thu, Aug 3, 2017 at 11:32 AM, Michael André Pearce <
> > >>>>> michael.andre.pearce@me.com> wrote:
> > >>>>>
> > >>>>>> We saw this too when running cluster mode and static discovery
> > before
> > >> we
> > >>>>>> moved to UDP and then finally went to single master cluster
due to
> > >> cost
> > >>>> in
> > >>>>>> some support licensing as had to reduce cpu counts.
> > >>>>>>
> > >>>>>> Sent from my iPhone
> > >>>>>>
> > >>>>>>> On 3 Aug 2017, at 17:31, Michael André Pearce <
> > >>>>>> michael.andre.pearce@me.com> wrote:
> > >>>>>>>
> > >>>>>>> The bit I'm getting at is it uses the discovery connection
when
> on
> > >>>>>> static instead of discovering getting the topology and
then using
> > that
> > >>>> to
> > >>>>>> make the connection.
> > >>>>>>>
> > >>>>>>> This is why when using topology and static you see
first two
> > >>>> connections
> > >>>>>> to same host as it uses the discovery connection first
which for
> > sake
> > >> of
> > >>>>>> discussing host a is first in list, and then the next uses
the
> > >> topology
> > >>>> one
> > >>>>>> where host a is first in list as such this is why it makes
to
> > >>>> connections
> > >>>>>> to host a before it makes one to host b.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Sent from my iPhone
> > >>>>>>>
> > >>>>>>>> On 3 Aug 2017, at 14:59, Clebert Suconic <
> > clebert.suconic@gmail.com
> > >>>
> > >>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>> On Thu, Aug 3, 2017 at 9:01 AM, Michael André
Pearce
> > >>>>>>>> <michael.andre.pearce@me.com> wrote:
> > >>>>>>>>> But what I'm saying is should it be that the
discovery should
> > >> happen
> > >>>>>> but then the real connection is made from the returned
topology.
> > Like
> > >>>> for
> > >>>>>> UDP instead of hoodwinking on the discovery connection.
> > >>>>>>>>
> > >>>>>>>> I'm not understanding your point here? the connection
factory
> will
> > >>>>>>>> always receive a list of the topology (No matter
if UDP or TCP)
> > and
> > >>>>>>>> load balance based on the topology returned.
> > >>>>>>>> There are users using this as a feature.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> What I think is needed here is a simple property
to the
> connection
> > >>>>>>>> factory, like.... useTopololgy, connectOntTopology
(or please
> help
> > >> me
> > >>>>>>>> find a better name).
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> then you could connect with:
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> ActiveMQConnectionFactory factory = new
> > >>>>>>>> ActiveMQConnectionFactory((tcp://NODE1:61616,tcp://NODE2:
> > >>>>>> 61616)?blockOnDurableSend=false&useTopology=false");
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> I"m not sure if useTopology would make it clear..
I'm still
> > thinking
> > >>>>>>>> about a better name.
> > >>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> Sent from my iPhone
> > >>>>>>>>>
> > >>>>>>>>>> On 3 Aug 2017, at 12:52, Clebert Suconic
<
> > >> clebert.suconic@gmail.com
> > >>>>>
> > >>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> It is not a bug. People use this to feed
an initial list than
> > the
> > >>>>>> topology
> > >>>>>>>>>> could be much bigger.
> > >>>>>>>>>>
> > >>>>>>>>>> On Thu, Aug 3, 2017 at 1:18 AM Michael
André Pearce <
> > >>>>>>>>>> michael.andre.pearce@me.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> To me this sounds like a bug, where
you get two connections
> > >> because
> > >>>>>> you
> > >>>>>>>>>>> use two lists.
> > >>>>>>>>>>>
> > >>>>>>>>>>> as in why doesn't it use the topology
list straight away?
> Fair
> > >>>>>> enough for
> > >>>>>>>>>>> discovery of that topology is should
temporarily make a
> > >> connection
> > >>>>>> using
> > >>>>>>>>>>> the static connections, but it should
disconnect and
> reconnect
> > >>>> using
> > >>>>>> the
> > >>>>>>>>>>> topology. I.e. It should just discover
the topology using the
> > >>>> static
> > >>>>>>>>>>> discovery list.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Similar to udp discovery it simply
discovers then it uses the
> > >>>>>> topology
> > >>>>>>>>>>> returned.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Sent from my iPad
> > >>>>>>>>>>>
> > >>>>>>>>>>>> On 3 Aug 2017, at 04:59, Clebert
Suconic <
> > >>>> clebert.suconic@gmail.com
> > >>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> I agree we could add an option.
We could use the URI
> > parameters
> > >>>>>> Thought
> > >>>>>>>>>>> as
> > >>>>>>>>>>>> a beanUtils?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On Wed, Aug 2, 2017 at 11:36
PM Justin Bertram <
> > >>>>>> jbertram@redhat.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I agree there should be an
option to stick with the
> "initial"
> > >>>>>> connectors
> > >>>>>>>>>>>>> rather than being forced to
use the topology.  This would
> be
> > an
> > >>>>>> option
> > >>>>>>>>>>> on
> > >>>>>>>>>>>>> the Netty connector.  I think
"useTopology" (defaults to
> > true)
> > >>>>>> would be
> > >>>>>>>>>>> a
> > >>>>>>>>>>>>> good name.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Justin
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:28
AM, cjaniake <
> > >>>>>> christian.janiake@movile.com>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi there, I have been using
the ActiveMQ Artemis JMS
> > interface
> > >>>>>> without
> > >>>>>>>>>>>>>> JNDI.
> > >>>>>>>>>>>>>> We are not using server
discovery, we use static
> connectors
> > >>>>>> instead.
> > >>>>>>>>>>>>>> In the connection factory
configuration we supply a list
> of
> > >>>>>> hosts, that
> > >>>>>>>>>>>>> are
> > >>>>>>>>>>>>>> located on two different
datacenters, acting as two
> > different
> > >>>>>> clusters.
> > >>>>>>>>>>>>>> Using the RoundRobinConnectionLoadBalancingPolicy
we
> > expected
> > >>>> to
> > >>>>>>>>>>> connect
> > >>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>> every server on the list,
but that was not what happened.
> > >>>>>>>>>>>>>> Debugging the code we realized
that, after connecting to
> the
> > >>>> first
> > >>>>>>>>>>>>> (random)
> > >>>>>>>>>>>>>> host on the list, the Server
Locator do not use the
> initial
> > >>>>>> connectors
> > >>>>>>>>>>>>> list
> > >>>>>>>>>>>>>> anymore, it uses the received
topology for the next
> > >> connections.
> > >>>>>>>>>>>>>> We understand this might
be useful in simpler scenarios,
> but
> > >>>> this
> > >>>>>> is
> > >>>>>>>>>>> not
> > >>>>>>>>>>>>>> working for us.
> > >>>>>>>>>>>>>> On a sandbox environment
we have even tried to remove the
> > >>>> cluster
> > >>>>>>>>>>>>>> connection
> > >>>>>>>>>>>>>> configuration, for the
servers to act on a stadalone
> manner,
> > >> but
> > >>>>>> even
> > >>>>>>>>>>>>>> though
> > >>>>>>>>>>>>>> the server locator acts
the same way, receiving a
> "topology"
> > >> of
> > >>>>>> only
> > >>>>>>>>>>> one
> > >>>>>>>>>>>>>> node and restrict the next
connections this one host.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> There is a number of problems
and inneficiencies we see on
> > >> this
> > >>>>>>>>>>> approach.
> > >>>>>>>>>>>>>> If we have a cluster with
3 hosts for example, and we
> > declare
> > >>>>>> those on
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>> host list and get 3 connections
using the round robin
> > policy,
> > >> we
> > >>>>>> would
> > >>>>>>>>>>>>>> expect to get one connection
for each host. But that's not
> > >> what
> > >>>>>>>>>>> happens.
> > >>>>>>>>>>>>>> The
> > >>>>>>>>>>>>>> load balancing policy starts
iterating over one list (the
> > >>>> initial
> > >>>>>>>>>>>>> connector
> > >>>>>>>>>>>>>> list) and after the first
successfull connection it
> > continues
> > >>>>>> iterating
> > >>>>>>>>>>>>>> over
> > >>>>>>>>>>>>>> another list (the received
topology), so most of the time
> > you
> > >>>>>> would get
> > >>>>>>>>>>>>> two
> > >>>>>>>>>>>>>> connections to the same
host and none for one of them.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> In a scenario like we have
here, with two clusters in
> > >> different
> > >>>>>>>>>>>>> locations,
> > >>>>>>>>>>>>>> it is even worse.
> > >>>>>>>>>>>>>> We would like to know if
we there is an option other than
> > >>>>>> creating a
> > >>>>>>>>>>>>>> connection factory for
each host we want to use, and if we
> > can
> > >>>>>> propose
> > >>>>>>>>>>> an
> > >>>>>>>>>>>>>> improvement.
> > >>>>>>>>>>>>>> We are willing to contribute
with the development, if we
> > have
> > >> an
> > >>>>>>>>>>>>>> understanding on a possible
solution for that problem.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thank you very much.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> View this message in context:
http://activemq.2283324.n4.
> > >>>>>> nabble.com/
> > >>>>>>>>>>>>>> ActiveMQConnectionFactory-use-
> > initial-connectors-instead-of-
> > >>>>>>>>>>>>>> received-topology-tp4729166.html
> > >>>>>>>>>>>>>> Sent from the ActiveMQ
- User mailing list archive at
> > >>>> Nabble.com.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> Clebert Suconic
> > >>>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> Clebert Suconic
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Clebert Suconic
> > >>>>>>
> > >>>>
> > >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message