activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Bertram <jbert...@redhat.com>
Subject Re: ActiveMQConnectionFactory: use initial connectors instead of received topology
Date Thu, 03 Aug 2017 19:07:40 GMT
That looks like a good approach to me.


Justin

On Thu, Aug 3, 2017 at 2:02 PM, Michael André Pearce <
michael.andre.pearce@me.com> wrote:

> That's fair enough.
>
> Going with then the other option of skipping the node if it's next in the
> list. That works if the node comes back in the first position but you'll
> still end up with the same problem if it is in the second position.
>
> Eg initial node order:
>
> A B C
>
> Topology
>
> B A C
>
> First three connections would be
>
> A B A
>
> in that case.
>
> Could I suggest maybe enhance it a little so the node that made the
> initial connection is moved to the end of the list/array in the topology.
> As such you get a truer round robin.
>
> Eg initial static list:
> A B C
>
> Connection made on A.
>
> Topology returned:
>
> A B C
>
> Reorder it moving A to the end.
>
> B C A
>
> As such first three connections will be:
>
> A B C
>
> And after continue to be evenly round robin'd
>
> This way it's a small reorder, get a balanced number of connections and
> you don't waste that first connection.
>
> WDYT?
>
> Sent from my iPhone
>
> > On 3 Aug 2017, at 19:02, Justin Bertram <jbertram@redhat.com> wrote:
> >
> > As stated previously, I'm not in favor of using the initial connection
> only
> > for discovery as I think it's wasteful.
> >
> >
> > Justin
> >
> > On Thu, Aug 3, 2017 at 12:41 PM, Michael André Pearce <
> > michael.andre.pearce@me.com> wrote:
> >
> >> How much would it be for it to also only use the topology eg the
> solution
> >> that just uses the first connection for discovery only, and then
> topology
> >> for the actual connections, as so then at least support expanding the
> >> cluster also but also load balancing better the initial connections.
> That
> >> way can support all three options.
> >>
> >> Sent from my iPhone
> >>
> >>> On 3 Aug 2017, at 18:24, Justin Bertram <jbertram@redhat.com> wrote:
> >>>
> >>> I see your point.  I supposed I was focusing on the title of his
> message
> >>> (i.e. "use initial connectors instead of received topology") rather
> than
> >>> the the details in the description of his problem.
> >>>
> >>> FWIW, I plan on implementing a new connector parameter to support
> >> ignoring
> >>> the topology because I think it will be useful for this use-case as
> well
> >> as
> >>> some others I've run across.
> >>>
> >>>
> >>> Justin
> >>>
> >>> On Thu, Aug 3, 2017 at 11:45 AM, Michael André Pearce <
> >>> michael.andre.pearce@me.com> wrote:
> >>>
> >>>> From what I read the double connection due to using both lists is the
> >>>> underlying issue from the original mail.
> >>>>
> >>>> "
> >>>> There is a number of problems and inneficiencies we see on this
> >> approach.
> >>>> If we have a cluster with 3 hosts for example, and we declare those
on
> >> the
> >>>> host list and get 3 connections using the round robin policy, we would
> >>>> expect to get one connection for each host. But that's not what
> happens.
> >>>> The
> >>>> load balancing policy starts iterating over one list (the initial
> >> connector
> >>>> list) and after the first successfull connection it continues
> iterating
> >>>> over
> >>>> another list (the received topology), so most of the time you would
> get
> >> two
> >>>> connections to the same host and none for one of them.
> >>>> "
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On 3 Aug 2017, at 17:41, Justin Bertram <jbertram@redhat.com>
wrote:
> >>>>>
> >>>>> IMO the way to deal with this is to just add a bit of logic so you
> >> don't
> >>>>> get 2 consecutive connections to the same host.  Making a connection
> >> with
> >>>>> the static connectors, getting the topology, closing the connection,
> >> and
> >>>>> then making another connection with the topology is wasteful.
> >>>>>
> >>>>> In any case, an option not to use the topology at all would be
> useful.
> >>>>> That is, after all, what this thread is really about.
> >>>>>
> >>>>>
> >>>>> Justin
> >>>>>
> >>>>> On Thu, Aug 3, 2017 at 11:32 AM, Michael André Pearce <
> >>>>> michael.andre.pearce@me.com> wrote:
> >>>>>
> >>>>>> We saw this too when running cluster mode and static discovery
> before
> >> we
> >>>>>> moved to UDP and then finally went to single master cluster
due to
> >> cost
> >>>> in
> >>>>>> some support licensing as had to reduce cpu counts.
> >>>>>>
> >>>>>> Sent from my iPhone
> >>>>>>
> >>>>>>> On 3 Aug 2017, at 17:31, Michael André Pearce <
> >>>>>> michael.andre.pearce@me.com> wrote:
> >>>>>>>
> >>>>>>> The bit I'm getting at is it uses the discovery connection
when on
> >>>>>> static instead of discovering getting the topology and then
using
> that
> >>>> to
> >>>>>> make the connection.
> >>>>>>>
> >>>>>>> This is why when using topology and static you see first
two
> >>>> connections
> >>>>>> to same host as it uses the discovery connection first which
for
> sake
> >> of
> >>>>>> discussing host a is first in list, and then the next uses the
> >> topology
> >>>> one
> >>>>>> where host a is first in list as such this is why it makes to
> >>>> connections
> >>>>>> to host a before it makes one to host b.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Sent from my iPhone
> >>>>>>>
> >>>>>>>> On 3 Aug 2017, at 14:59, Clebert Suconic <
> clebert.suconic@gmail.com
> >>>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> On Thu, Aug 3, 2017 at 9:01 AM, Michael André Pearce
> >>>>>>>> <michael.andre.pearce@me.com> wrote:
> >>>>>>>>> But what I'm saying is should it be that the discovery
should
> >> happen
> >>>>>> but then the real connection is made from the returned topology.
> Like
> >>>> for
> >>>>>> UDP instead of hoodwinking on the discovery connection.
> >>>>>>>>
> >>>>>>>> I'm not understanding your point here? the connection
factory will
> >>>>>>>> always receive a list of the topology (No matter if
UDP or TCP)
> and
> >>>>>>>> load balance based on the topology returned.
> >>>>>>>> There are users using this as a feature.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> What I think is needed here is a simple property to
the connection
> >>>>>>>> factory, like.... useTopololgy, connectOntTopology (or
please help
> >> me
> >>>>>>>> find a better name).
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> then you could connect with:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> ActiveMQConnectionFactory factory = new
> >>>>>>>> ActiveMQConnectionFactory((tcp://NODE1:61616,tcp://NODE2:
> >>>>>> 61616)?blockOnDurableSend=false&useTopology=false");
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> I"m not sure if useTopology would make it clear.. I'm
still
> thinking
> >>>>>>>> about a better name.
> >>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Sent from my iPhone
> >>>>>>>>>
> >>>>>>>>>> On 3 Aug 2017, at 12:52, Clebert Suconic <
> >> clebert.suconic@gmail.com
> >>>>>
> >>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> It is not a bug. People use this to feed an
initial list than
> the
> >>>>>> topology
> >>>>>>>>>> could be much bigger.
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Aug 3, 2017 at 1:18 AM Michael André
Pearce <
> >>>>>>>>>> michael.andre.pearce@me.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> To me this sounds like a bug, where you
get two connections
> >> because
> >>>>>> you
> >>>>>>>>>>> use two lists.
> >>>>>>>>>>>
> >>>>>>>>>>> as in why doesn't it use the topology list
straight away? Fair
> >>>>>> enough for
> >>>>>>>>>>> discovery of that topology is should temporarily
make a
> >> connection
> >>>>>> using
> >>>>>>>>>>> the static connections, but it should disconnect
and reconnect
> >>>> using
> >>>>>> the
> >>>>>>>>>>> topology. I.e. It should just discover the
topology using the
> >>>> static
> >>>>>>>>>>> discovery list.
> >>>>>>>>>>>
> >>>>>>>>>>> Similar to udp discovery it simply discovers
then it uses the
> >>>>>> topology
> >>>>>>>>>>> returned.
> >>>>>>>>>>>
> >>>>>>>>>>> Sent from my iPad
> >>>>>>>>>>>
> >>>>>>>>>>>> On 3 Aug 2017, at 04:59, Clebert Suconic
<
> >>>> clebert.suconic@gmail.com
> >>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> I agree we could add an option. We could
use the URI
> parameters
> >>>>>> Thought
> >>>>>>>>>>> as
> >>>>>>>>>>>> a beanUtils?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Aug 2, 2017 at 11:36 PM
Justin Bertram <
> >>>>>> jbertram@redhat.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I agree there should be an option
to stick with the "initial"
> >>>>>> connectors
> >>>>>>>>>>>>> rather than being forced to use
the topology.  This would be
> an
> >>>>>> option
> >>>>>>>>>>> on
> >>>>>>>>>>>>> the Netty connector.  I think "useTopology"
(defaults to
> true)
> >>>>>> would be
> >>>>>>>>>>> a
> >>>>>>>>>>>>> good name.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Justin
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Aug 2, 2017 at 9:28 AM,
cjaniake <
> >>>>>> christian.janiake@movile.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi there, I have been using
the ActiveMQ Artemis JMS
> interface
> >>>>>> without
> >>>>>>>>>>>>>> JNDI.
> >>>>>>>>>>>>>> We are not using server discovery,
we use static connectors
> >>>>>> instead.
> >>>>>>>>>>>>>> In the connection factory configuration
we supply a list of
> >>>>>> hosts, that
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>>> located on two different datacenters,
acting as two
> different
> >>>>>> clusters.
> >>>>>>>>>>>>>> Using the RoundRobinConnectionLoadBalancingPolicy
we
> expected
> >>>> to
> >>>>>>>>>>> connect
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>> every server on the list, but
that was not what happened.
> >>>>>>>>>>>>>> Debugging the code we realized
that, after connecting to the
> >>>> first
> >>>>>>>>>>>>> (random)
> >>>>>>>>>>>>>> host on the list, the Server
Locator do not use the initial
> >>>>>> connectors
> >>>>>>>>>>>>> list
> >>>>>>>>>>>>>> anymore, it uses the received
topology for the next
> >> connections.
> >>>>>>>>>>>>>> We understand this might be
useful in simpler scenarios, but
> >>>> this
> >>>>>> is
> >>>>>>>>>>> not
> >>>>>>>>>>>>>> working for us.
> >>>>>>>>>>>>>> On a sandbox environment we
have even tried to remove the
> >>>> cluster
> >>>>>>>>>>>>>> connection
> >>>>>>>>>>>>>> configuration, for the servers
to act on a stadalone manner,
> >> but
> >>>>>> even
> >>>>>>>>>>>>>> though
> >>>>>>>>>>>>>> the server locator acts the
same way, receiving a "topology"
> >> of
> >>>>>> only
> >>>>>>>>>>> one
> >>>>>>>>>>>>>> node and restrict the next connections
this one host.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> There is a number of problems
and inneficiencies we see on
> >> this
> >>>>>>>>>>> approach.
> >>>>>>>>>>>>>> If we have a cluster with 3
hosts for example, and we
> declare
> >>>>>> those on
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>> host list and get 3 connections
using the round robin
> policy,
> >> we
> >>>>>> would
> >>>>>>>>>>>>>> expect to get one connection
for each host. But that's not
> >> what
> >>>>>>>>>>> happens.
> >>>>>>>>>>>>>> The
> >>>>>>>>>>>>>> load balancing policy starts
iterating over one list (the
> >>>> initial
> >>>>>>>>>>>>> connector
> >>>>>>>>>>>>>> list) and after the first successfull
connection it
> continues
> >>>>>> iterating
> >>>>>>>>>>>>>> over
> >>>>>>>>>>>>>> another list (the received topology),
so most of the time
> you
> >>>>>> would get
> >>>>>>>>>>>>> two
> >>>>>>>>>>>>>> connections to the same host
and none for one of them.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In a scenario like we have here,
with two clusters in
> >> different
> >>>>>>>>>>>>> locations,
> >>>>>>>>>>>>>> it is even worse.
> >>>>>>>>>>>>>> We would like to know if we
there is an option other than
> >>>>>> creating a
> >>>>>>>>>>>>>> connection factory for each
host we want to use, and if we
> can
> >>>>>> propose
> >>>>>>>>>>> an
> >>>>>>>>>>>>>> improvement.
> >>>>>>>>>>>>>> We are willing to contribute
with the development, if we
> have
> >> an
> >>>>>>>>>>>>>> understanding on a possible
solution for that problem.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thank you very much.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> View this message in context:
http://activemq.2283324.n4.
> >>>>>> nabble.com/
> >>>>>>>>>>>>>> ActiveMQConnectionFactory-use-
> initial-connectors-instead-of-
> >>>>>>>>>>>>>> received-topology-tp4729166.html
> >>>>>>>>>>>>>> Sent from the ActiveMQ - User
mailing list archive at
> >>>> Nabble.com.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Clebert Suconic
> >>>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Clebert Suconic
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Clebert Suconic
> >>>>>>
> >>>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message