activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Clebert Suconic <clebert.suco...@gmail.com>
Subject Re: [DISCUSS] Artemis addressing improvements, JMS component removal and potential 2.0.0
Date Wed, 07 Dec 2016 18:25:33 GMT
I have rebased ARTEMIS-780 in top of master. There was a lot of conflicts...

I have aggregated/squashed most of the commits by chronological order
almost. So if Martyn had 10 commits in series I had squashed all of
them, since they were small comments anyways. The good thing about
this is that nobody would lose authorship of these commits.

We will need to come up with more meaningful messages for these
commits before we can merge into master. But this is getting into a
very good shape. I'm impressed by the amount of work I see done on
this branch. Very well done guys! I mean it!

Also, I have saved the old branch before I pushed -f into my fork as
old-ARTEMIS-780 in case I broke anything on the process. Please check
everything and let me know if I did.


And please rebase more often on this branch unless you merge it soon.


On Mon, Nov 28, 2016 at 2:36 PM, Clebert Suconic
<clebert.suconic@gmail.com> wrote:
> If / when we do the 2.0 bump, I would like to move a few classes.
> Mainly under server.impl... I would like to move activations under a
> package for activation, replicationendpoints for a package for
> replications...    some small stuff like that just to reorganize
> little things like this a bit.
>
> We can't do that now as that would break API and compatibility, but if
> we do the bump, I would like to make that simple move.
>
> On Thu, Nov 24, 2016 at 4:41 AM, Martyn Taylor <mtaylor@redhat.com> wrote:
>> Hi Matt,
>>
>> Comments inline.
>>
>> On Mon, Nov 21, 2016 at 7:02 PM, Matt Pavlovich <mattrpav@gmail.com> wrote:
>>
>>> Martyn-
>>>
>>> I think you nailed it here-- well done =)
>>>
>>> My notes in-line--
>>>
>>> On 11/21/16 10:45 AM, Martyn Taylor wrote:
>>>
>>>> 1. Ability to route messages to queues with the same address, but
>>>> different
>>>> routing semantics.
>>>>
>>>> The proposal outlined in ARTEMIS-780 outlines a new model that introduces
>>>> an address object at the configuration and management layer. In the
>>>> proposal it is not possible to create 2 addresses with different routing
>>>> types. This causes a problem with existing clients (JMS, STOMP and for
>>>> compatability with other vendors).
>>>>
>>>> Potential Modification: Addresses can have multiple routing type
>>>> “endpoints”, either “multicast” only, “anycast” only or both.
The example
>>>> below would be used to represent a JMS Topic called “foo”, with a single
>>>> subscription queue and a JMS Queue called “foo”. N.B. The actual XML
is
>>>> just an example, there are multiple ways this could be represented that we
>>>> can define later.
>>>>
>>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>>> <*queues*>            <*queue* *name**="**foo”* />        
</*queues*>
>>>>       </*anycast*>      <*mulcast*>         <*queues*>
>>>> <*queue* *name**="my.topic.subscription" */>         </*queues*>
>>>> </*multicast*>   </*address*></*addresses*>
>>>>
>>> I think this solves it. The crux of the issues (for me) boils down to
>>> auto-creation of destinations across protocols. Having this show up in the
>>> configs would give developers and admins more information to troubleshoot
>>> the mixed address type+protocol scenario.
>>>
>>> 2. Sending to “multicast”, “anycast” or “all”
>>>>
>>>> As mentioned earlier JMS (and other clients such as STOMP via prefixing)
>>>> allow the producer to identify the type of end point it would like to send
>>>> to.
>>>>
>>>> If a JMS client creates a producer and passes in a topic with address
>>>> “foo”. Then only the queues associated with the “multicast” section
of the
>>>> address. A similar thing happens when the JMS producer sends to a “queue”
>>>> messages should be distributed amongst the queues associated with the
>>>> “anycast” section of the address.
>>>>
>>>> There may also be a case when a producer does not identify the endpoint
>>>> type, and simply sends to “foo”. AMQP or MQTT may want to do this. In
this
>>>> scenario both should happen. All the queues under the multicast section
>>>> get
>>>> a copy of the message, and one queue under the anycast section gets the
>>>> message.
>>>>
>>>> Modification: None Needed. Internal APIs would need to be updated to allow
>>>> this functionality.
>>>>
>>> I think the "deliver to all" scenario should be fine. This seems analogous
>>> to a CompositeDestination in ActiveMQ 5.x. I'll map through some scenarios
>>> and report back any gotchas.
>>>
>>> 3. Support for prefixes to identify endpoint types
>>>>
>>>> Many clients, ActiveMQ 5.x, STOMP and potential clients from alternate
>>>> vendors, identify the endpoint type (in producer and consumer) using a
>>>> prefix notation.
>>>>
>>>> e.g. queue:///foo
>>>>
>>>> Which would identify:
>>>>
>>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>>> <*queues*>            <*queue* *name**="my.foo.queue" */>
>>>> </*queues*>      </*anycast*>   </*address*></*addresses*>
>>>>
>>>> Modifications Needed: None to the model. An additional parameter to the
>>>> acceptors should be added to identify the prefix.
>>>>
>>> Just as a check point in the syntax+naming convention in your provided
>>> example... would the name actually be:
>>>
>>> <*queue* *name**="foo" .. vs "my.foo.queue" ?
>>>
>> The queue name can be anything.  It's the address that is used by
>> consumer/producer.  The protocol handler / broker will decided which queue
>> to connect to.
>>
>>>
>>> 4. Multiple endpoints are defined, but client does not specify “endpoint
>>>> routing type” when consuming
>>>>
>>>> Handling cases where consumers does not pass enough information in their
>>>> address or via protocol specific mechanisms to identify an endpoint. Let’s
>>>> say an AMQP client, requests to subscribe to the address “foo”, but passes
>>>> no extra information. In the cases where there are only a single endpoint
>>>> type defined, the consumer would associated with that endpoint type.
>>>> However, when both endpoint types are defined, the protocol handler does
>>>> not know whether to associate this consumer with a queue under the
>>>> “anycast” section, or whether to create a new queue under the “multicast”
>>>> section. e.g.
>>>>
>>>> Consume: “foo”
>>>>
>>>> <*addresses*>
>>>>
>>>>     <*address* *name**="foo"*>      <*anycast*>         <*queues*>
>>>>        <*queue* *name**="**foo”* />         </*queues*>
>>>> </*anycast*>      <*multicast*>         <*queues*>    
       <*queue*
>>>> *name**="my.topic.subscription" */>         </*queues*>
>>>> </*multicast*>   </*address*></*addresses*>
>>>>
>>>> In this scenario, we can make the default configurable on the
>>>> protocol/acceptor. Possible options for this could be:
>>>>
>>>> “multicast”: Defaults multicast
>>>>
>>>> “anycast”: Defaults to anycast
>>>>
>>>> “error”: Returns an error to the client
>>>>
>>>> Alternatively each protocol handler could handle this in the most sensible
>>>> way for that protocol. MQTT might default to “multicast”, STOMP “anycast”,
>>>> and AMQP to “error”.
>>>>
>>>
>>> Yep, this works great. I think there are two flags on the acceptors.. one
>>> for auto-create and one for default handling of name collision. The
>>> defaults would most likely be the same.
>>>
>>> Something along the lines of:
>>> auto-create-default = "multicast | anycast"
>>> no-prefix-default = "multicast | anycast | error"
>>>
>>> 5. Fully qualified address names
>>>>
>>>> This feature allows a client to identify a particular address on a
>>>> specific
>>>> broker in a cluster. This could be achieved by the client using some form
>>>> of address as:
>>>>
>>>> queue:///host/broker/address/
>>>>
>>>> Matt could you elaborate on the drivers behind this requirement please.
>>>>
>>>> I am of the opinion that this is out of the scope of the addressing
>>>> changes, and is more to do with redirecting in cluster scenarios. The
>>>> current model will support this address syntax if we want to use it in the
>>>> future.
>>>>
>>> I agree that tackling the impl of this should be out-of-scope. My
>>> recommendation is to consider it in addressing now, so we can hopefully
>>> avoid any breakage down the road.
>>>
>>> A widely used feature in other EMS brokers (IBM MQ, Tibco EMS, etc) is the
>>> ability to fully address a destination using a format similar to this:
>>>
>>> queue://brokerB/myQueue
>>>
>> The advantage of this is to allow for scaling of the number of destinations
>>> and allows for more dynamic broker networks to be created without
>>> applications having to have connection information for all brokers in a
>>> broker network. Think simple delivery+routing, and not horizontal scaling.
>>> It is very analogous to SMTP mail routing.
>>>
>>> Producer behavior:
>>>
>>> 1. Client X connects to brokerA and sends it a message addressed:
>>> queue://brokerB/myQueue
>>> 2. brokerA accepts the message on behalf of brokerB and handles all
>>> acknowledgement and persistence accordingly
>>> 3. brokerA would then store the message in a "queue" for brokerB. Note:
>>> All messages for brokerB are generally stored in one queue-- this is how it
>>> helps with destination scaling
>>>
>>> Broker to broker behavior:
>>>
>>> There are generally two scenarios: always-on or periodic-check
>>>
>>> In "always-on"
>>> 1. brokerA looks for a brokerB in its list of cluster connections and then
>>> sends all messages for all queues for brokerB (or brokerB pulls all
>>> messages, depending on cluster connection config)
>>>
>>> In "periodic-check"
>>> 1. brokerB connects to brokerA (or vice-versa) on a given time interval
>>> and then receives any messages that have arrived since last check
>>>
>>> TL;DR;
>>>
>>> It would be cool to consider remote broker delivery for messages while
>>> refactoring the address handling code. This would bring Artemis inline with
>>> the rest of the commercial EMS brokers. The impact now, hopefully, is minor
>>> and just thinking about default prefixes.
>>>
>> Understood, from our conversations on IRC I can see why this might be
>> useful.
>>
>>>
>>> Thanks,
>>> -Matt
>>>
>>>
>>>
>
>
>
> --
> Clebert Suconic



-- 
Clebert Suconic

Mime
View raw message