activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Justin Bertram <jbert...@redhat.com>
Subject Re: ActiveMQConnectionFactory: use initial connectors instead of received topology
Date Thu, 03 Aug 2017 18:02:07 GMT
As stated previously, I'm not in favor of using the initial connection only
for discovery as I think it's wasteful.


Justin

On Thu, Aug 3, 2017 at 12:41 PM, Michael André Pearce <
michael.andre.pearce@me.com> wrote:

> How much would it be for it to also only use the topology eg the solution
> that just uses the first connection for discovery only, and then topology
> for the actual connections, as so then at least support expanding the
> cluster also but also load balancing better the initial connections. That
> way can support all three options.
>
> Sent from my iPhone
>
> > On 3 Aug 2017, at 18:24, Justin Bertram <jbertram@redhat.com> wrote:
> >
> > I see your point.  I supposed I was focusing on the title of his message
> > (i.e. "use initial connectors instead of received topology") rather than
> > the the details in the description of his problem.
> >
> > FWIW, I plan on implementing a new connector parameter to support
> ignoring
> > the topology because I think it will be useful for this use-case as well
> as
> > some others I've run across.
> >
> >
> > Justin
> >
> > On Thu, Aug 3, 2017 at 11:45 AM, Michael André Pearce <
> > michael.andre.pearce@me.com> wrote:
> >
> >> From what I read the double connection due to using both lists is the
> >> underlying issue from the original mail.
> >>
> >> "
> >> There is a number of problems and inneficiencies we see on this
> approach.
> >> If we have a cluster with 3 hosts for example, and we declare those on
> the
> >> host list and get 3 connections using the round robin policy, we would
> >> expect to get one connection for each host. But that's not what happens.
> >> The
> >> load balancing policy starts iterating over one list (the initial
> connector
> >> list) and after the first successfull connection it continues iterating
> >> over
> >> another list (the received topology), so most of the time you would get
> two
> >> connections to the same host and none for one of them.
> >> "
> >>
> >> Sent from my iPhone
> >>
> >>> On 3 Aug 2017, at 17:41, Justin Bertram <jbertram@redhat.com> wrote:
> >>>
> >>> IMO the way to deal with this is to just add a bit of logic so you
> don't
> >>> get 2 consecutive connections to the same host.  Making a connection
> with
> >>> the static connectors, getting the topology, closing the connection,
> and
> >>> then making another connection with the topology is wasteful.
> >>>
> >>> In any case, an option not to use the topology at all would be useful.
> >>> That is, after all, what this thread is really about.
> >>>
> >>>
> >>> Justin
> >>>
> >>> On Thu, Aug 3, 2017 at 11:32 AM, Michael André Pearce <
> >>> michael.andre.pearce@me.com> wrote:
> >>>
> >>>> We saw this too when running cluster mode and static discovery before
> we
> >>>> moved to UDP and then finally went to single master cluster due to
> cost
> >> in
> >>>> some support licensing as had to reduce cpu counts.
> >>>>
> >>>> Sent from my iPhone
> >>>>
> >>>>> On 3 Aug 2017, at 17:31, Michael André Pearce <
> >>>> michael.andre.pearce@me.com> wrote:
> >>>>>
> >>>>> The bit I'm getting at is it uses the discovery connection when
on
> >>>> static instead of discovering getting the topology and then using that
> >> to
> >>>> make the connection.
> >>>>>
> >>>>> This is why when using topology and static you see first two
> >> connections
> >>>> to same host as it uses the discovery connection first which for sake
> of
> >>>> discussing host a is first in list, and then the next uses the
> topology
> >> one
> >>>> where host a is first in list as such this is why it makes to
> >> connections
> >>>> to host a before it makes one to host b.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Sent from my iPhone
> >>>>>
> >>>>>> On 3 Aug 2017, at 14:59, Clebert Suconic <clebert.suconic@gmail.com
> >
> >>>> wrote:
> >>>>>>
> >>>>>> On Thu, Aug 3, 2017 at 9:01 AM, Michael André Pearce
> >>>>>> <michael.andre.pearce@me.com> wrote:
> >>>>>>> But what I'm saying is should it be that the discovery should
> happen
> >>>> but then the real connection is made from the returned topology. Like
> >> for
> >>>> UDP instead of hoodwinking on the discovery connection.
> >>>>>>
> >>>>>> I'm not understanding your point here? the connection factory
will
> >>>>>> always receive a list of the topology (No matter if UDP or TCP)
and
> >>>>>> load balance based on the topology returned.
> >>>>>> There are users using this as a feature.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> What I think is needed here is a simple property to the connection
> >>>>>> factory, like.... useTopololgy, connectOntTopology (or please
help
> me
> >>>>>> find a better name).
> >>>>>>
> >>>>>>
> >>>>>> then you could connect with:
> >>>>>>
> >>>>>>
> >>>>>> ActiveMQConnectionFactory factory = new
> >>>>>> ActiveMQConnectionFactory((tcp://NODE1:61616,tcp://NODE2:
> >>>> 61616)?blockOnDurableSend=false&useTopology=false");
> >>>>>>
> >>>>>>
> >>>>>> I"m not sure if useTopology would make it clear.. I'm still
thinking
> >>>>>> about a better name.
> >>>>>>
> >>>>>>>
> >>>>>>> Sent from my iPhone
> >>>>>>>
> >>>>>>>> On 3 Aug 2017, at 12:52, Clebert Suconic <
> clebert.suconic@gmail.com
> >>>
> >>>> wrote:
> >>>>>>>>
> >>>>>>>> It is not a bug. People use this to feed an initial
list than the
> >>>> topology
> >>>>>>>> could be much bigger.
> >>>>>>>>
> >>>>>>>> On Thu, Aug 3, 2017 at 1:18 AM Michael André Pearce
<
> >>>>>>>> michael.andre.pearce@me.com> wrote:
> >>>>>>>>
> >>>>>>>>> To me this sounds like a bug, where you get two
connections
> because
> >>>> you
> >>>>>>>>> use two lists.
> >>>>>>>>>
> >>>>>>>>> as in why doesn't it use the topology list straight
away? Fair
> >>>> enough for
> >>>>>>>>> discovery of that topology is should temporarily
make a
> connection
> >>>> using
> >>>>>>>>> the static connections, but it should disconnect
and reconnect
> >> using
> >>>> the
> >>>>>>>>> topology. I.e. It should just discover the topology
using the
> >> static
> >>>>>>>>> discovery list.
> >>>>>>>>>
> >>>>>>>>> Similar to udp discovery it simply discovers then
it uses the
> >>>> topology
> >>>>>>>>> returned.
> >>>>>>>>>
> >>>>>>>>> Sent from my iPad
> >>>>>>>>>
> >>>>>>>>>> On 3 Aug 2017, at 04:59, Clebert Suconic <
> >> clebert.suconic@gmail.com
> >>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I agree we could add an option. We could use
the URI parameters
> >>>> Thought
> >>>>>>>>> as
> >>>>>>>>>> a beanUtils?
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On Wed, Aug 2, 2017 at 11:36 PM Justin Bertram
<
> >>>> jbertram@redhat.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> I agree there should be an option to stick
with the "initial"
> >>>> connectors
> >>>>>>>>>>> rather than being forced to use the topology.
 This would be an
> >>>> option
> >>>>>>>>> on
> >>>>>>>>>>> the Netty connector.  I think "useTopology"
(defaults to true)
> >>>> would be
> >>>>>>>>> a
> >>>>>>>>>>> good name.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Justin
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Aug 2, 2017 at 9:28 AM, cjaniake
<
> >>>> christian.janiake@movile.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi there, I have been using the ActiveMQ
Artemis JMS interface
> >>>> without
> >>>>>>>>>>>> JNDI.
> >>>>>>>>>>>> We are not using server discovery, we
use static connectors
> >>>> instead.
> >>>>>>>>>>>> In the connection factory configuration
we supply a list of
> >>>> hosts, that
> >>>>>>>>>>> are
> >>>>>>>>>>>> located on two different datacenters,
acting as two different
> >>>> clusters.
> >>>>>>>>>>>> Using the RoundRobinConnectionLoadBalancingPolicy
we expected
> >> to
> >>>>>>>>> connect
> >>>>>>>>>>>> to
> >>>>>>>>>>>> every server on the list, but that was
not what happened.
> >>>>>>>>>>>> Debugging the code we realized that,
after connecting to the
> >> first
> >>>>>>>>>>> (random)
> >>>>>>>>>>>> host on the list, the Server Locator
do not use the initial
> >>>> connectors
> >>>>>>>>>>> list
> >>>>>>>>>>>> anymore, it uses the received topology
for the next
> connections.
> >>>>>>>>>>>> We understand this might be useful in
simpler scenarios, but
> >> this
> >>>> is
> >>>>>>>>> not
> >>>>>>>>>>>> working for us.
> >>>>>>>>>>>> On a sandbox environment we have even
tried to remove the
> >> cluster
> >>>>>>>>>>>> connection
> >>>>>>>>>>>> configuration, for the servers to act
on a stadalone manner,
> but
> >>>> even
> >>>>>>>>>>>> though
> >>>>>>>>>>>> the server locator acts the same way,
receiving a "topology"
> of
> >>>> only
> >>>>>>>>> one
> >>>>>>>>>>>> node and restrict the next connections
this one host.
> >>>>>>>>>>>>
> >>>>>>>>>>>> There is a number of problems and inneficiencies
we see on
> this
> >>>>>>>>> approach.
> >>>>>>>>>>>> If we have a cluster with 3 hosts for
example, and we declare
> >>>> those on
> >>>>>>>>>>> the
> >>>>>>>>>>>> host list and get 3 connections using
the round robin policy,
> we
> >>>> would
> >>>>>>>>>>>> expect to get one connection for each
host. But that's not
> what
> >>>>>>>>> happens.
> >>>>>>>>>>>> The
> >>>>>>>>>>>> load balancing policy starts iterating
over one list (the
> >> initial
> >>>>>>>>>>> connector
> >>>>>>>>>>>> list) and after the first successfull
connection it continues
> >>>> iterating
> >>>>>>>>>>>> over
> >>>>>>>>>>>> another list (the received topology),
so most of the time you
> >>>> would get
> >>>>>>>>>>> two
> >>>>>>>>>>>> connections to the same host and none
for one of them.
> >>>>>>>>>>>>
> >>>>>>>>>>>> In a scenario like we have here, with
two clusters in
> different
> >>>>>>>>>>> locations,
> >>>>>>>>>>>> it is even worse.
> >>>>>>>>>>>> We would like to know if we there is
an option other than
> >>>> creating a
> >>>>>>>>>>>> connection factory for each host we
want to use, and if we can
> >>>> propose
> >>>>>>>>> an
> >>>>>>>>>>>> improvement.
> >>>>>>>>>>>> We are willing to contribute with the
development, if we have
> an
> >>>>>>>>>>>> understanding on a possible solution
for that problem.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thank you very much.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> View this message in context: http://activemq.2283324.n4.
> >>>> nabble.com/
> >>>>>>>>>>>> ActiveMQConnectionFactory-use-initial-connectors-instead-of-
> >>>>>>>>>>>> received-topology-tp4729166.html
> >>>>>>>>>>>> Sent from the ActiveMQ - User mailing
list archive at
> >> Nabble.com.
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Clebert Suconic
> >>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Clebert Suconic
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Clebert Suconic
> >>>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message