activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Bish <tabish...@gmail.com>
Subject Re: [DISCUSS] Artemis addressing improvements, JMS component removal and potential 2.0.0
Date Wed, 07 Dec 2016 18:52:31 GMT
This would be a good time to move to master, would allow other to more 
easily get onboard

On 12/07/2016 01:25 PM, Clebert Suconic wrote:
> I have rebased ARTEMIS-780 in top of master. There was a lot of conflicts...
>
> I have aggregated/squashed most of the commits by chronological order
> almost. So if Martyn had 10 commits in series I had squashed all of
> them, since they were small comments anyways. The good thing about
> this is that nobody would lose authorship of these commits.
>
> We will need to come up with more meaningful messages for these
> commits before we can merge into master. But this is getting into a
> very good shape. I'm impressed by the amount of work I see done on
> this branch. Very well done guys! I mean it!
>
> Also, I have saved the old branch before I pushed -f into my fork as
> old-ARTEMIS-780 in case I broke anything on the process. Please check
> everything and let me know if I did.
>
>
> And please rebase more often on this branch unless you merge it soon.
>
>
> On Mon, Nov 28, 2016 at 2:36 PM, Clebert Suconic
> <clebert.suconic@gmail.com> wrote:
>> If / when we do the 2.0 bump, I would like to move a few classes.
>> Mainly under server.impl... I would like to move activations under a
>> package for activation, replicationendpoints for a package for
>> replications...    some small stuff like that just to reorganize
>> little things like this a bit.
>>
>> We can't do that now as that would break API and compatibility, but if
>> we do the bump, I would like to make that simple move.
>>
>> On Thu, Nov 24, 2016 at 4:41 AM, Martyn Taylor <mtaylor@redhat.com> wrote:
>>> Hi Matt,
>>>
>>> Comments inline.
>>>
>>> On Mon, Nov 21, 2016 at 7:02 PM, Matt Pavlovich <mattrpav@gmail.com> wrote:
>>>
>>>> Martyn-
>>>>
>>>> I think you nailed it here-- well done =)
>>>>
>>>> My notes in-line--
>>>>
>>>> On 11/21/16 10:45 AM, Martyn Taylor wrote:
>>>>
>>>>> 1. Ability to route messages to queues with the same address, but
>>>>> different
>>>>> routing semantics.
>>>>>
>>>>> The proposal outlined in ARTEMIS-780 outlines a new model that introduces
>>>>> an address object at the configuration and management layer. In the
>>>>> proposal it is not possible to create 2 addresses with different routing
>>>>> types. This causes a problem with existing clients (JMS, STOMP and for
>>>>> compatability with other vendors).
>>>>>
>>>>> Potential Modification: Addresses can have multiple routing type
>>>>> “endpoints”, either “multicast” only, “anycast” only or both.
The example
>>>>> below would be used to represent a JMS Topic called “foo”, with a
single
>>>>> subscription queue and a JMS Queue called “foo”. N.B. The actual
XML is
>>>>> just an example, there are multiple ways this could be represented that
we
>>>>> can define later.
>>>>>
>>>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>>>> <*queues*>            <*queue* *name**="**foo”* />    
    </*queues*>
>>>>>        </*anycast*>      <*mulcast*>         <*queues*>
>>>>> <*queue* *name**="my.topic.subscription" */>         </*queues*>
>>>>> </*multicast*>   </*address*></*addresses*>
>>>>>
>>>> I think this solves it. The crux of the issues (for me) boils down to
>>>> auto-creation of destinations across protocols. Having this show up in the
>>>> configs would give developers and admins more information to troubleshoot
>>>> the mixed address type+protocol scenario.
>>>>
>>>> 2. Sending to “multicast”, “anycast” or “all”
>>>>> As mentioned earlier JMS (and other clients such as STOMP via prefixing)
>>>>> allow the producer to identify the type of end point it would like to
send
>>>>> to.
>>>>>
>>>>> If a JMS client creates a producer and passes in a topic with address
>>>>> “foo”. Then only the queues associated with the “multicast” section
of the
>>>>> address. A similar thing happens when the JMS producer sends to a “queue”
>>>>> messages should be distributed amongst the queues associated with the
>>>>> “anycast” section of the address.
>>>>>
>>>>> There may also be a case when a producer does not identify the endpoint
>>>>> type, and simply sends to “foo”. AMQP or MQTT may want to do this.
In this
>>>>> scenario both should happen. All the queues under the multicast section
>>>>> get
>>>>> a copy of the message, and one queue under the anycast section gets the
>>>>> message.
>>>>>
>>>>> Modification: None Needed. Internal APIs would need to be updated to
allow
>>>>> this functionality.
>>>>>
>>>> I think the "deliver to all" scenario should be fine. This seems analogous
>>>> to a CompositeDestination in ActiveMQ 5.x. I'll map through some scenarios
>>>> and report back any gotchas.
>>>>
>>>> 3. Support for prefixes to identify endpoint types
>>>>> Many clients, ActiveMQ 5.x, STOMP and potential clients from alternate
>>>>> vendors, identify the endpoint type (in producer and consumer) using
a
>>>>> prefix notation.
>>>>>
>>>>> e.g. queue:///foo
>>>>>
>>>>> Which would identify:
>>>>>
>>>>> <*addresses*>   <*address* *name**="foo"*>      <*anycast*>
>>>>> <*queues*>            <*queue* *name**="my.foo.queue" */>
>>>>> </*queues*>      </*anycast*>   </*address*></*addresses*>
>>>>>
>>>>> Modifications Needed: None to the model. An additional parameter to the
>>>>> acceptors should be added to identify the prefix.
>>>>>
>>>> Just as a check point in the syntax+naming convention in your provided
>>>> example... would the name actually be:
>>>>
>>>> <*queue* *name**="foo" .. vs "my.foo.queue" ?
>>>>
>>> The queue name can be anything.  It's the address that is used by
>>> consumer/producer.  The protocol handler / broker will decided which queue
>>> to connect to.
>>>
>>>> 4. Multiple endpoints are defined, but client does not specify “endpoint
>>>>> routing type” when consuming
>>>>>
>>>>> Handling cases where consumers does not pass enough information in their
>>>>> address or via protocol specific mechanisms to identify an endpoint.
Let’s
>>>>> say an AMQP client, requests to subscribe to the address “foo”, but
passes
>>>>> no extra information. In the cases where there are only a single endpoint
>>>>> type defined, the consumer would associated with that endpoint type.
>>>>> However, when both endpoint types are defined, the protocol handler does
>>>>> not know whether to associate this consumer with a queue under the
>>>>> “anycast” section, or whether to create a new queue under the “multicast”
>>>>> section. e.g.
>>>>>
>>>>> Consume: “foo”
>>>>>
>>>>> <*addresses*>
>>>>>
>>>>>      <*address* *name**="foo"*>      <*anycast*>        
<*queues*>
>>>>>         <*queue* *name**="**foo”* />         </*queues*>
>>>>> </*anycast*>      <*multicast*>         <*queues*>
           <*queue*
>>>>> *name**="my.topic.subscription" */>         </*queues*>
>>>>> </*multicast*>   </*address*></*addresses*>
>>>>>
>>>>> In this scenario, we can make the default configurable on the
>>>>> protocol/acceptor. Possible options for this could be:
>>>>>
>>>>> “multicast”: Defaults multicast
>>>>>
>>>>> “anycast”: Defaults to anycast
>>>>>
>>>>> “error”: Returns an error to the client
>>>>>
>>>>> Alternatively each protocol handler could handle this in the most sensible
>>>>> way for that protocol. MQTT might default to “multicast”, STOMP “anycast”,
>>>>> and AMQP to “error”.
>>>>>
>>>> Yep, this works great. I think there are two flags on the acceptors.. one
>>>> for auto-create and one for default handling of name collision. The
>>>> defaults would most likely be the same.
>>>>
>>>> Something along the lines of:
>>>> auto-create-default = "multicast | anycast"
>>>> no-prefix-default = "multicast | anycast | error"
>>>>
>>>> 5. Fully qualified address names
>>>>> This feature allows a client to identify a particular address on a
>>>>> specific
>>>>> broker in a cluster. This could be achieved by the client using some
form
>>>>> of address as:
>>>>>
>>>>> queue:///host/broker/address/
>>>>>
>>>>> Matt could you elaborate on the drivers behind this requirement please.
>>>>>
>>>>> I am of the opinion that this is out of the scope of the addressing
>>>>> changes, and is more to do with redirecting in cluster scenarios. The
>>>>> current model will support this address syntax if we want to use it in
the
>>>>> future.
>>>>>
>>>> I agree that tackling the impl of this should be out-of-scope. My
>>>> recommendation is to consider it in addressing now, so we can hopefully
>>>> avoid any breakage down the road.
>>>>
>>>> A widely used feature in other EMS brokers (IBM MQ, Tibco EMS, etc) is the
>>>> ability to fully address a destination using a format similar to this:
>>>>
>>>> queue://brokerB/myQueue
>>>>
>>> The advantage of this is to allow for scaling of the number of destinations
>>>> and allows for more dynamic broker networks to be created without
>>>> applications having to have connection information for all brokers in a
>>>> broker network. Think simple delivery+routing, and not horizontal scaling.
>>>> It is very analogous to SMTP mail routing.
>>>>
>>>> Producer behavior:
>>>>
>>>> 1. Client X connects to brokerA and sends it a message addressed:
>>>> queue://brokerB/myQueue
>>>> 2. brokerA accepts the message on behalf of brokerB and handles all
>>>> acknowledgement and persistence accordingly
>>>> 3. brokerA would then store the message in a "queue" for brokerB. Note:
>>>> All messages for brokerB are generally stored in one queue-- this is how
it
>>>> helps with destination scaling
>>>>
>>>> Broker to broker behavior:
>>>>
>>>> There are generally two scenarios: always-on or periodic-check
>>>>
>>>> In "always-on"
>>>> 1. brokerA looks for a brokerB in its list of cluster connections and then
>>>> sends all messages for all queues for brokerB (or brokerB pulls all
>>>> messages, depending on cluster connection config)
>>>>
>>>> In "periodic-check"
>>>> 1. brokerB connects to brokerA (or vice-versa) on a given time interval
>>>> and then receives any messages that have arrived since last check
>>>>
>>>> TL;DR;
>>>>
>>>> It would be cool to consider remote broker delivery for messages while
>>>> refactoring the address handling code. This would bring Artemis inline with
>>>> the rest of the commercial EMS brokers. The impact now, hopefully, is minor
>>>> and just thinking about default prefixes.
>>>>
>>> Understood, from our conversations on IRC I can see why this might be
>>> useful.
>>>
>>>> Thanks,
>>>> -Matt
>>>>
>>>>
>>>>
>>
>>
>> --
>> Clebert Suconic
>
>


-- 
Tim Bish
twitter: @tabish121
blog: http://timbish.blogspot.com/


Mime
View raw message