activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Pavlovich <mattr...@gmail.com>
Subject Re: [DISCUSS] Artemis addressing improvements, JMS component removal and potential 2.0.0
Date Wed, 07 Dec 2016 20:01:01 GMT
+1


On 12/7/16 1:52 PM, Timothy Bish wrote:
> This would be a good time to move to master, would allow other to more 
> easily get onboard
>
> On 12/07/2016 01:25 PM, Clebert Suconic wrote:
>> I have rebased ARTEMIS-780 in top of master. There was a lot of 
>> conflicts...
>>
>> I have aggregated/squashed most of the commits by chronological order
>> almost. So if Martyn had 10 commits in series I had squashed all of
>> them, since they were small comments anyways. The good thing about
>> this is that nobody would lose authorship of these commits.
>>
>> We will need to come up with more meaningful messages for these
>> commits before we can merge into master. But this is getting into a
>> very good shape. I'm impressed by the amount of work I see done on
>> this branch. Very well done guys! I mean it!
>>
>> Also, I have saved the old branch before I pushed -f into my fork as
>> old-ARTEMIS-780 in case I broke anything on the process. Please check
>> everything and let me know if I did.
>>
>>
>> And please rebase more often on this branch unless you merge it soon.
>>
>>
>> On Mon, Nov 28, 2016 at 2:36 PM, Clebert Suconic
>> <clebert.suconic@gmail.com> wrote:
>>> If / when we do the 2.0 bump, I would like to move a few classes.
>>> Mainly under server.impl... I would like to move activations under a
>>> package for activation, replicationendpoints for a package for
>>> replications...    some small stuff like that just to reorganize
>>> little things like this a bit.
>>>
>>> We can't do that now as that would break API and compatibility, but if
>>> we do the bump, I would like to make that simple move.
>>>
>>> On Thu, Nov 24, 2016 at 4:41 AM, Martyn Taylor <mtaylor@redhat.com> 
>>> wrote:
>>>> Hi Matt,
>>>>
>>>> Comments inline.
>>>>
>>>> On Mon, Nov 21, 2016 at 7:02 PM, Matt Pavlovich 
>>>> <mattrpav@gmail.com> wrote:
>>>>
>>>>> Martyn-
>>>>>
>>>>> I think you nailed it here-- well done =)
>>>>>
>>>>> My notes in-line--
>>>>>
>>>>> On 11/21/16 10:45 AM, Martyn Taylor wrote:
>>>>>
>>>>>> 1. Ability to route messages to queues with the same address, but
>>>>>> different
>>>>>> routing semantics.
>>>>>>
>>>>>> The proposal outlined in ARTEMIS-780 outlines a new model that 
>>>>>> introduces
>>>>>> an address object at the configuration and management layer. In the
>>>>>> proposal it is not possible to create 2 addresses with different

>>>>>> routing
>>>>>> types. This causes a problem with existing clients (JMS, STOMP 
>>>>>> and for
>>>>>> compatability with other vendors).
>>>>>>
>>>>>> Potential Modification: Addresses can have multiple routing type
>>>>>> “endpoints”, either “multicast” only, “anycast” only
or both. The 
>>>>>> example
>>>>>> below would be used to represent a JMS Topic called “foo”, with
a 
>>>>>> single
>>>>>> subscription queue and a JMS Queue called “foo”. N.B. The actual

>>>>>> XML is
>>>>>> just an example, there are multiple ways this could be 
>>>>>> represented that we
>>>>>> can define later.
>>>>>>
>>>>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>>>>> <*queues*>            <*queue* *name**="**foo”* />
        
>>>>>> </*queues*>
>>>>>>        </*anycast*>      <*mulcast*> <*queues*>
>>>>>> <*queue* *name**="my.topic.subscription" */>         </*queues*>
>>>>>> </*multicast*> </*address*></*addresses*>
>>>>>>
>>>>> I think this solves it. The crux of the issues (for me) boils down to
>>>>> auto-creation of destinations across protocols. Having this show 
>>>>> up in the
>>>>> configs would give developers and admins more information to 
>>>>> troubleshoot
>>>>> the mixed address type+protocol scenario.
>>>>>
>>>>> 2. Sending to “multicast”, “anycast” or “all”
>>>>>> As mentioned earlier JMS (and other clients such as STOMP via 
>>>>>> prefixing)
>>>>>> allow the producer to identify the type of end point it would 
>>>>>> like to send
>>>>>> to.
>>>>>>
>>>>>> If a JMS client creates a producer and passes in a topic with 
>>>>>> address
>>>>>> “foo”. Then only the queues associated with the “multicast”

>>>>>> section of the
>>>>>> address. A similar thing happens when the JMS producer sends to a

>>>>>> “queue”
>>>>>> messages should be distributed amongst the queues associated with

>>>>>> the
>>>>>> “anycast” section of the address.
>>>>>>
>>>>>> There may also be a case when a producer does not identify the 
>>>>>> endpoint
>>>>>> type, and simply sends to “foo”. AMQP or MQTT may want to do

>>>>>> this. In this
>>>>>> scenario both should happen. All the queues under the multicast 
>>>>>> section
>>>>>> get
>>>>>> a copy of the message, and one queue under the anycast section 
>>>>>> gets the
>>>>>> message.
>>>>>>
>>>>>> Modification: None Needed. Internal APIs would need to be updated

>>>>>> to allow
>>>>>> this functionality.
>>>>>>
>>>>> I think the "deliver to all" scenario should be fine. This seems 
>>>>> analogous
>>>>> to a CompositeDestination in ActiveMQ 5.x. I'll map through some 
>>>>> scenarios
>>>>> and report back any gotchas.
>>>>>
>>>>> 3. Support for prefixes to identify endpoint types
>>>>>> Many clients, ActiveMQ 5.x, STOMP and potential clients from 
>>>>>> alternate
>>>>>> vendors, identify the endpoint type (in producer and consumer) 
>>>>>> using a
>>>>>> prefix notation.
>>>>>>
>>>>>> e.g. queue:///foo
>>>>>>
>>>>>> Which would identify:
>>>>>>
>>>>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>>>>> <*queues*>            <*queue* *name**="my.foo.queue" */>
>>>>>> </*queues*>      </*anycast*> </*address*></*addresses*>
>>>>>>
>>>>>> Modifications Needed: None to the model. An additional parameter

>>>>>> to the
>>>>>> acceptors should be added to identify the prefix.
>>>>>>
>>>>> Just as a check point in the syntax+naming convention in your 
>>>>> provided
>>>>> example... would the name actually be:
>>>>>
>>>>> <*queue* *name**="foo" .. vs "my.foo.queue" ?
>>>>>
>>>> The queue name can be anything.  It's the address that is used by
>>>> consumer/producer.  The protocol handler / broker will decided 
>>>> which queue
>>>> to connect to.
>>>>
>>>>> 4. Multiple endpoints are defined, but client does not specify 
>>>>> “endpoint
>>>>>> routing type” when consuming
>>>>>>
>>>>>> Handling cases where consumers does not pass enough information 
>>>>>> in their
>>>>>> address or via protocol specific mechanisms to identify an 
>>>>>> endpoint. Let’s
>>>>>> say an AMQP client, requests to subscribe to the address “foo”,

>>>>>> but passes
>>>>>> no extra information. In the cases where there are only a single

>>>>>> endpoint
>>>>>> type defined, the consumer would associated with that endpoint type.
>>>>>> However, when both endpoint types are defined, the protocol 
>>>>>> handler does
>>>>>> not know whether to associate this consumer with a queue under the
>>>>>> “anycast” section, or whether to create a new queue under the

>>>>>> “multicast”
>>>>>> section. e.g.
>>>>>>
>>>>>> Consume: “foo”
>>>>>>
>>>>>> <*addresses*>
>>>>>>
>>>>>>      <*address* *name**="foo"*> <*anycast*>         <*queues*>
>>>>>>         <*queue* *name**="**foo”* /> </*queues*>
>>>>>> </*anycast*>      <*multicast*> <*queues*>    
       <*queue*
>>>>>> *name**="my.topic.subscription" */> </*queues*>
>>>>>> </*multicast*> </*address*></*addresses*>
>>>>>>
>>>>>> In this scenario, we can make the default configurable on the
>>>>>> protocol/acceptor. Possible options for this could be:
>>>>>>
>>>>>> “multicast”: Defaults multicast
>>>>>>
>>>>>> “anycast”: Defaults to anycast
>>>>>>
>>>>>> “error”: Returns an error to the client
>>>>>>
>>>>>> Alternatively each protocol handler could handle this in the most

>>>>>> sensible
>>>>>> way for that protocol. MQTT might default to “multicast”, STOMP

>>>>>> “anycast”,
>>>>>> and AMQP to “error”.
>>>>>>
>>>>> Yep, this works great. I think there are two flags on the 
>>>>> acceptors.. one
>>>>> for auto-create and one for default handling of name collision. The
>>>>> defaults would most likely be the same.
>>>>>
>>>>> Something along the lines of:
>>>>> auto-create-default = "multicast | anycast"
>>>>> no-prefix-default = "multicast | anycast | error"
>>>>>
>>>>> 5. Fully qualified address names
>>>>>> This feature allows a client to identify a particular address on
a
>>>>>> specific
>>>>>> broker in a cluster. This could be achieved by the client using 
>>>>>> some form
>>>>>> of address as:
>>>>>>
>>>>>> queue:///host/broker/address/
>>>>>>
>>>>>> Matt could you elaborate on the drivers behind this requirement 
>>>>>> please.
>>>>>>
>>>>>> I am of the opinion that this is out of the scope of the addressing
>>>>>> changes, and is more to do with redirecting in cluster scenarios.

>>>>>> The
>>>>>> current model will support this address syntax if we want to use

>>>>>> it in the
>>>>>> future.
>>>>>>
>>>>> I agree that tackling the impl of this should be out-of-scope. My
>>>>> recommendation is to consider it in addressing now, so we can 
>>>>> hopefully
>>>>> avoid any breakage down the road.
>>>>>
>>>>> A widely used feature in other EMS brokers (IBM MQ, Tibco EMS, 
>>>>> etc) is the
>>>>> ability to fully address a destination using a format similar to 
>>>>> this:
>>>>>
>>>>> queue://brokerB/myQueue
>>>>>
>>>> The advantage of this is to allow for scaling of the number of 
>>>> destinations
>>>>> and allows for more dynamic broker networks to be created without
>>>>> applications having to have connection information for all brokers 
>>>>> in a
>>>>> broker network. Think simple delivery+routing, and not horizontal 
>>>>> scaling.
>>>>> It is very analogous to SMTP mail routing.
>>>>>
>>>>> Producer behavior:
>>>>>
>>>>> 1. Client X connects to brokerA and sends it a message addressed:
>>>>> queue://brokerB/myQueue
>>>>> 2. brokerA accepts the message on behalf of brokerB and handles all
>>>>> acknowledgement and persistence accordingly
>>>>> 3. brokerA would then store the message in a "queue" for brokerB. 
>>>>> Note:
>>>>> All messages for brokerB are generally stored in one queue-- this 
>>>>> is how it
>>>>> helps with destination scaling
>>>>>
>>>>> Broker to broker behavior:
>>>>>
>>>>> There are generally two scenarios: always-on or periodic-check
>>>>>
>>>>> In "always-on"
>>>>> 1. brokerA looks for a brokerB in its list of cluster connections 
>>>>> and then
>>>>> sends all messages for all queues for brokerB (or brokerB pulls all
>>>>> messages, depending on cluster connection config)
>>>>>
>>>>> In "periodic-check"
>>>>> 1. brokerB connects to brokerA (or vice-versa) on a given time 
>>>>> interval
>>>>> and then receives any messages that have arrived since last check
>>>>>
>>>>> TL;DR;
>>>>>
>>>>> It would be cool to consider remote broker delivery for messages 
>>>>> while
>>>>> refactoring the address handling code. This would bring Artemis 
>>>>> inline with
>>>>> the rest of the commercial EMS brokers. The impact now, hopefully, 
>>>>> is minor
>>>>> and just thinking about default prefixes.
>>>>>
>>>> Understood, from our conversations on IRC I can see why this might be
>>>> useful.
>>>>
>>>>> Thanks,
>>>>> -Matt
>>>>>
>>>>>
>>>>>
>>>
>>>
>>> -- 
>>> Clebert Suconic
>>
>>
>
>


Mime
View raw message