Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
MIME-Version: 1.0
In-Reply-To: <ad415253-8622-5aa7-7af6-b06f818d8303@gmail.com>
References: <CAH6wpnpKUHTjLcquVzfH5t09PdxxcZG4zSyJ_vgjK4-fkKKExA@mail.gmail.com>
 <ad415253-8622-5aa7-7af6-b06f818d8303@gmail.com>
From: Martyn Taylor <mtaylor@redhat.com>
Date: Thu, 17 Nov 2016 10:49:50 +0000
Message-ID: <CAH6wpnqq6wb7Ds3YrNOgv9c7Z3TDuXKRGVsXCc7s_WOyibiPRw@mail.gmail.com>
Subject: Re: [DISCUSS] Artemis addressing improvements, JMS component removal
 and potential 2.0.0
To: dev@activemq.apache.org
Content-Type: multipart/alternative; boundary=94eb2c036346eba8d505417cf253
archived-at: Thu, 17 Nov 2016 10:50:03 -0000

--94eb2c036346eba8d505417cf253
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

This is great feedback Matt, thanks.  I have a couple of questions/comments
below:

On Wed, Nov 16, 2016 at 6:23 PM, Matt Pavlovich <mattrpav@gmail.com> wrote:

> Hi Martin-
>
> Glad to see this area getting dedicated attention. A couple things I
> didn't see covered in the doc or the JIRA comments. (I'll be adding to th=
e
> JIRA comments as well.)
>
> Items:
>
> 0. Pre-configuring destinations is a big brain drain, so anything that ca=
n
> be client-driven is a win. Also, protocol specific handlers could perform
> the admin operations on-demand.
>
>    For example:  session.createDurableSubscriber(...)   The JMS handler
> create the subscription on the behalf of the client.
>
Yes I agree.  We need to ensure we can both ways of defining the endpoint
semantics, i.e. allow clients to request endpoint requirements and also
have broker side configuration drive endpoint behaviour, ideally using the
same underlying mechanism.

>
> 1. Separate topic and queue namespaces.. in JMS topic:///foo !=3D
> queue:///foo. The addressing will need some sort of way to separate the t=
wo
> during naming collisions.
>
Just so I understand exactly what you are saying here.  You're saying that
a client sends to "foo" and a consumer received messages sent to "foo".  In
order for the consumer to consume from "foo" it passes in either "foo",
"queue:///foo" or "topic:///foo" which determines how the messages are
propagated to the client?  "foo" means let the broker decide,
"queue:///foo" and "topic:///foo" mean let the client decide.  In addition
to these two approaches, it may be that the protocol itself wants to
decide.  MQTT for example, always requires a subscription.

One way to do this, not straying too far from the original proposal, would
be to make the address uniqueness a combination of the routing type and the
address name.  This would allow something like:

<address name=3D"foo" routingType=3D"anycast">
<address name=3D"foo" routingType=3D"multicast">

We'd need to ensure there is a precedent set for times when a subscriber
just subscribes to "foo".  I'd say it makes sense for "multicast" to take
precedence in this case.

I think it probably makes the most sense to have the following precedence
for the deciding party:

1. Broker
2. Address prefixing/name scheme
3. Protocol

I think the prefix also needs to be configurable, but "queue:///"
"topic:///" seems like a sensible default.

>
> 2. In ActiveMQ 5.x, AMQP and STOMP handled the addressing by using
> queue:/// and topic:/// prefixes. I don't think that is necessarily a bad
> thing, but something to consider b/c we need to support #1
>
+1

>
> 3. As far as destination behaviors, how about using uri parameters to pas=
s
> provider (Artemis) specific settings on-the-fly?
>
>     For example:  in AMPQ the address could be
> topic:///foo?type=3DnonSharedNonDurable etc..  same for MQTT, STOMP, etc.


>     There is precedence in using uri parameters to configure the
> Destination in JMS as well. IBM MQ has session.createQueue("My.Queue?
> targetClient=3D1")
>
>     Note: AMQP supports options as well, so that could be used as well.
> However, uri's tend to be better for externalizing configuration manageme=
nt.
>
I think supporting both options, uri and protocol specific parameters is
useful.  Rather than "nonSharedDurable" I think I'd prefer to map these
things onto address properties.  For example:

"topic://foo?maxConsumers=3D1"

Where the "topic:///" prefix is configurable.  This is essentially a non
shared, durable subscription.

>
> 4. Destination name separation b/w protocol handlers.  STOMP and MQTT lik=
e
> "/" and JMS likes "." as destination name separators. Is there any though=
t
> to having a native destination name separator and then have
> protocol-specific convertors?
>
This is how it works right now.  We have a native path separator which is
".".  Protocol handlers map subscription addresses down to this.  This does
mean that to define a multicast address for MQTT, you would need to do:

"foo.bar" (vs "foo/bar" which is protocol specific).

I've also outlined in the proposal a goal to allow these path separators to
be configurable.  So you can specify pathSeparator=3D"/", "." which would
mean that you can configure "foo/bar" or "foo.bar" they'd both act in the
same way.

>
> 5. Fully qualified destination names that include a broker name. Other
> providers support fully-qualified destination names in JMS following the
> format:  queue://$brokerName/$queueName. Adding this would go a long way
> to supporting migration of current applications without having to change
> client-code.
>
This is a little differently to how clustering currently works, I think we
need to give this one some more thought.  Right now queues are clustered
automatically (well providing you enable the correct address namespace for
the cluster connection).  If you have a client listening on broker2 and a
producer from broker1, the messages will get propagated across the cluster.

You may have already explained this to me in the past, but can you give me
an example use case of when this might be necessary.

>
>     Note: This would probably impact cluster handling as well, so perhaps
> in phase 1 there is just a placeholder for supporting a broker name in th=
e
> future?
>
> -Matt

Thanks
Martyn

>
>
> On 11/16/16 10:16 AM, Martyn Taylor wrote:
>
>> All,
>>
>> Some discussion has happened around this topic already, but I wanted to
>> ensure that everyone here, who have not been following the
>> JIRA/ARTEMIS-780
>> branch has a chance for input and to digest the information in this
>> proposal.
>>
>> In order to understand the motivators outlined here, you first need to
>> understand how the existing addressing model works in Artemis. For those
>> of
>> you who are not familiar with how things currently work, I=E2=80=99ve ad=
ded a
>> document to the ARTEMIS-780 JIRA in the attachments section, that gives =
an
>> overview of the existing model and some more detail / examples of the
>> proposal: *https://issues.apache.org/jira/browse/ARTEMIS-780
>> <https://issues.apache.org/jira/browse/ARTEMIS-780>*
>>
>> To summarise here, the Artemis routing/addressing model has some
>> restrictions:
>>
>> 1. It=E2=80=99s not possible with core (and therefore across all protoco=
ls) to
>> define ,at the broker side, semantics about addresses. i.e. whether an
>> address behaves as a =E2=80=9Cpoint to point=E2=80=9D or =E2=80=9Cpublis=
h subscribe=E2=80=9D end point
>>
>> 2. For JMS destinations additional configuration and objects were added =
to
>> the broker, that rely on name-spacing to add semantics to addresses i.e.
>> =E2=80=9Cjms.topic.=E2=80=9D =E2=80=9Cjms.queue.=E2=80=9D  A couple of i=
ssues with this:
>>
>>     1.
>>
>>     This only works for JMS and no other protocols
>>     2.
>>
>>     Name-spacing causes issues for cross protocol communication
>>     3.
>>
>>     It means there=E2=80=99s two ways of doing things, 1 for JMS and 1 f=
or
>>     everything else.
>>
>> 3. The JMS and Core destination definitions do not have enough informati=
on
>> to define more intricate behaviours. Such as whether an address should
>> behave like a =E2=80=9Cshared subscription=E2=80=9D or similar to a =E2=
=80=9Cvolatile
>> subscription=E2=80=9D
>> where clients don=E2=80=99t get messages missed when they are offline.
>>
>> 4. Some protocols (AMQP is a good example) don=E2=80=99t have enough inf=
ormation
>> in
>> their frames for the broker to determine how to behave for certain
>> endpoints and rely on broker side configuration (or provider specific
>> parameters).
>>
>> Proposal
>>
>> What I=E2=80=99d like to do (and what I=E2=80=99ve proposed in ARTEMIS-7=
80) is to get rid
>> of the JMS specific components and create a single unified mechanism for
>> configuring all types of endpoints across all protocols to define:
>>
>>     -
>>
>>     Point to point (queue)
>>     -
>>
>>     Shared Durable Subscriptions
>>     -
>>
>>     Shared Non Durable Subscriptions
>>     -
>>
>>     Non Shared durable subscriptions
>>     -
>>
>>
>>     Non Shared Non durable subscriptions
>>
>> To do this, the idea is to create a new =E2=80=9CAddress=E2=80=9D config=
uration/management
>> object, that has certain properties such as a routing type which
>> represents
>> how messages are routed to queues with this address.
>>
>> When a request for subscription is received by Artemis, the relevant pie=
ce
>> can just look up the address and check it=E2=80=99s properties to determ=
ine how to
>> behave (or if an address doesn=E2=80=99t exist) then default to our exis=
ting
>> behaviour. For those interested in the details of how this might work I=
=E2=80=99ve
>> outlined some specific examples in the document on the JIRA.
>>
>> What are the user impacts:
>>
>> 1. Configuration would need to be revised in order to expose the new
>> addressing object. I propose that we either continue supporting the old
>> schema for a while and/or provide a tool to migrate the configuration
>> schema.
>>
>> 2. Some new management operations would need to be added to expose the n=
ew
>> objects.
>>
>> 3. The JMS configuration and management objects would become obsolete, a=
nd
>> would need removing. The Broker side JMS resources were only a thin faca=
de
>> to allow some JMS specific behaviour for managing destinations and for
>> things like registering objects in JNDI.
>>
>> Broker side JNDI was removed in Artemis 1.0 in order to align with
>> ActiveMQ
>> 5.x style of client side JNDI.  These JMS pieces and their management
>> objects don't really do much, creating connection factories for instance
>> offers no functionality right now.  Going forward, users should be able =
to
>> use the core management API to do everything.
>>
>> 4. All client applications should behave exactly as they were before. Th=
e
>> proposal is for adding features to the core model, not removing any.  Fo=
r
>> things like the Artemis JMS client which relied on name-spaces, they=E2=
=80=99ll be
>> a mechanism to define a name-spaced address and a mechanism to switch ba=
ck
>> on name-spaces in the client.
>>
>> 5. Given some of the API changes and removal of the JMS specific pieces.
>> This would likely warrant a major bump. i.e. Artemis 2.0.0.
>>
>> Whilst I=E2=80=99ve been looking at this, it=E2=80=99s become apparent, =
that the JMS
>> pieces
>> have leaked into lots of areas of the code base, which does mean we=E2=
=80=99d need
>> to do a fair amount refactoring to move these bits to the new model.
>>
>> In my opinion this proposal can only be a good thing. It creates a singl=
e
>> place (core) where all addressing objects are configured and managed and
>> allows all protocol managers to plug into the same mechanism. It solves
>> some of the cross protocol JMS =E2=86=92 other protocols that we=E2=80=
=99ve seen users
>> struggle with, but still offers a way to support all the old behaviour i=
n
>> client applications.
>>
>> What are others thoughts on this? Any suggestions, comments or concerns?
>>
>> Regards
>> Martyn
>>
>>
>

--94eb2c036346eba8d505417cf253--