activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francesco PADOVANI <Francesco.PADOV...@bticino.it>
Subject Re: Apache Artemis - Stress-test time
Date Fri, 21 Apr 2017 10:04:45 GMT
Martyn, sorry, one more thing,
for the previous test we didn't use security: we assigned the same role to all user and the
role had all permissions on all addresses/queues (#).
Now we'd like to try by using security, but we're facing an issue when a client publish on
a topic which another client has subscribed: the latter is disconnected due to permission
violation error:

"
09:55:58,213 DEBUG [org.apache.activemq.artemis.core.protocol.mqtt] Error processing Control
Packet, Disconnecting Client: ActiveMQSecurityException[errorType=SECU
RITY_EXCEPTION message=AMQ119032: User: 38316 does not have permission='CONSUME' on address
$sys.mqtt.queue.qos2.franzsub.$sys.mqtt.queue.qos2.franzsub]
        at org.apache.activemq.artemis.core.security.impl.SecurityStoreImpl.check(SecurityStoreImpl.java:201)
[artemis-server-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSHOT]
        at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.securityCheck(ServerSessionImpl.java:403)
[artemis-server-2.1.0-SNAPSHOT.jar:2.1.0-SNAPSH
OT]
        at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.createConsumer(ServerSessionImpl.java:441)
[artemis-server-2.1.0-SNAPSHOT.jar:2.1.0-SNAPS
HOT]
        at org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.createManagementConsumer(MQTTPublishManager.java:86)
[artemis-mqtt-protocol-2.1.0-SNAP
SHOT.jar:]
        at org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.getManagementAddress(MQTTPublishManager.java:192)
[artemis-mqtt-protocol-2.1.0-SNAPSHO
T.jar:]
        at org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.handlePubRec(MQTTPublishManager.java:201)
[artemis-mqtt-protocol-2.1.0-SNAPSHOT.jar:]
        at org.apache.activemq.artemis.core.protocol.mqtt.MQTTProtocolHandler.handlePubrec(MQTTProtocolHandler.java:235)
[artemis-mqtt-protocol-2.1.0-SNAPSHOT.jar:
]
...
"

I see, you already resolved a similar bug (ARTEMIS-990, "User: Customer does not have permission='CREATE_DURABLE_QUEUE'
on address $sys.mqtt.queue.qos2"). Nevertheles, we have the above problem on version we are
using (2.1.o build starting from master, commit 310f953c790380b65d2f02081a1888309dc93ab5).
And the below workaround

<security-setting match="$sys.mqtt.#">
<permission type="createDurableQueue" roles="<MQTT Client Roles Here>"/>
<permission type="deleteDurableQueue" roles="<MQTT Client Roles Here>"/>
<permission type="consume" roles="<MQTT Client Roles Here>"/>
<permission type="send" roles="<MQTT Client Roles Here>"/>
</security-setting>

seems not working.
Any suggestion? Where I'm wrong?

Thanks

Francesco





________________________________
From: Francesco PADOVANI <Francesco.PADOVANI@bticino.it>
Sent: Wednesday, April 19, 2017 2:20:58 PM
To: users@activemq.apache.org
Subject: Re: Apache Artemis - Stress-test time

Hi Martyn,
I can confirm you that once we have all our clients connected, the system performs as
normal.

We've done some more tests.
First of all we've disabled our custom plugin for managing redis authentication/authorization.
So we started from a 2.1.0 clean installation (build from hash commit 2bcc255f4ca85c179671bb4bd4f0e77bd09e3281).
By testing our usual use case (subscriber clients who subscribe 50 topics each one) we reach
a serious performance problem around 8000 connections.
Instead, by trying with subscriber clients who make a single subscription each one (using
the wildcard char '#') it seems there are no problems: we've reached 16000 connections (8000
pub and 8000 sub) in about 5 minutes (the time to start clients) with system resources always
under control.
It seems that subscriptions are the key: or I'm wrong?
If so, is there a way to optimize the subscription phase? Maybe also by adjusting our configuration
settings?

Thanks as usual.

Francesco

________________________________
From: Martyn Taylor <mtaylor@redhat.com>
Sent: Tuesday, April 11, 2017 12:49:57 PM
To: users@activemq.apache.org
Subject: Re: Apache Artemis - Stress-test time

Clebert: Yes we could optimize on this area of the code to avoid the
bindings query.  To determine whether or not this is the root cause of the
problem experience by Francesco and Alessandro, we just need to follow the
same test without using retained messages.

Francesco, Alessandro is this something you could test for us?

Could you also let us know if staggering the subscribe helps.  Like try
testing connecting 100 clients at a time, wait a while then attach another
100.  Once you have all your clients connected does the system perform as
normal?  (this will help us determine whether or not the bottleneck is the
subscribe or just normal function with that many consumers).

Another quick q, are your subscriptions matching most messages.  e.g. are
you using "#" or similar.  If using "#" or subscriptions that match most
messages I'd expect the performance to drop, since the broker needs to
iterate through every subscription queue.  If you have 20,000 subscription
queues matching every message you might see perf decrease.

Any more information on your use case would be helpful in diagnosing the
bottleneck

Thanks




On Mon, Apr 10, 2017 at 5:33 PM, Clebert Suconic <clebert.suconic@gmail.com>
wrote:

> only thing I see on the thread dump is reained message is doing a
> query on the queue every time:
>
>  - org.apache.activemq.artemis.api.core.SimpleString.split(char)
> @bci=100, line=314 (Compiled frame)
>
>  - org.apache.activemq.artemis.core.postoffice.impl.
> AddressImpl.<init>(org.apache.activemq.artemis.api.core.SimpleString,
> org.apache.activemq.artemis.core.config.WildcardConfiguration)
> @bci=31, line=48 (Compiled frame)
>
>  - org.apache.activemq.artemis.core.postoffice.impl.SimpleAddressManager.
> getMatchingBindings(org.apache.activemq.artemis.api.core.SimpleString)
> @bci=77, line=121 (Compiled frame)
>
>  - org.apache.activemq.artemis.core.postoffice.impl.PostOfficeImpl.
> getMatchingBindings(org.apache.activemq.artemis.api.core.SimpleString)
> @bci=5, line=656 (Compiled frame)
>
>  - org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.
> bindingQuery(org.apache.activemq.artemis.api.core.SimpleString)
> @bci=110, line=722 (Compiled frame)
>
>  - org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.
> executeBindingQuery(org.apache.activemq.artemis.api.core.SimpleString)
> @bci=9, line=730 (Compiled frame)
>
>  - org.apache.activemq.artemis.core.protocol.mqtt.
> MQTTRetainMessageManager.addRetainedMessagesToQueue(
> org.apache.activemq.artemis.core.server.Queue,
> java.lang.String) @bci=27, line=74 (Compiled frame)
>
>  -
>
>
> @Mtaylor: any way to avoid it?
>
> On Mon, Apr 10, 2017 at 11:42 AM, alessandro.zannini@bticino.it
> <alessandro.zannini@bticino.it> wrote:
> > Here attached the output of "jstack -F" command.
> > jstack.zip <http://activemq.2283324.n4.nabble.com/file/n4724787/
> jstack.zip>
> >
> >
> >
> > --
> > View this message in context: http://activemq.2283324.n4.
> nabble.com/Apache-Artemis-Stress-test-time-tp4723999p4724787.html
> > Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>
>
>
> --
> Clebert Suconic
>

________________________________

Ce message, ainsi que tous les fichiers joints ? ce message, peuvent contenir des informations
sensibles et/ ou confidentielles ne devant pas ?tre divulgu?es. Si vous n'?tes pas le destinataire
de ce message (ou que vous recevez ce message par erreur), nous vous remercions de le notifier
imm?diatement ? son exp?diteur, et de d?truire ce message. Toute copie, divulgation, modification,
utilisation ou diffusion, non autoris?e, directe ou indirecte, de tout ou partie de ce message,
est strictement interdite.


This e-mail, and any document attached hereby, may contain confidential and/or privileged
information. If you are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail. Any unauthorized, direct or
indirect, copying, disclosure, distribution or other use of the material or parts thereof
is strictly forbidden.

________________________________

Ce message, ainsi que tous les fichiers joints ? ce message, peuvent contenir des informations
sensibles et/ ou confidentielles ne devant pas ?tre divulgu?es. Si vous n'?tes pas le destinataire
de ce message (ou que vous recevez ce message par erreur), nous vous remercions de le notifier
imm?diatement ? son exp?diteur, et de d?truire ce message. Toute copie, divulgation, modification,
utilisation ou diffusion, non autoris?e, directe ou indirecte, de tout ou partie de ce message,
est strictement interdite.


This e-mail, and any document attached hereby, may contain confidential and/or privileged
information. If you are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and destroy this e-mail. Any unauthorized, direct or
indirect, copying, disclosure, distribution or other use of the material or parts thereof
is strictly forbidden.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message