From users-return-51742-apmail-activemq-users-archive=activemq.apache.org@activemq.apache.org Mon Aug 12 13:41:58 2019 Return-Path: X-Original-To: apmail-activemq-users-archive@www.apache.org Delivered-To: apmail-activemq-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by minotaur.apache.org (Postfix) with SMTP id 7062E19E2A for ; Mon, 12 Aug 2019 13:41:58 +0000 (UTC) Received: (qmail 5788 invoked by uid 500); 12 Aug 2019 13:41:57 -0000 Delivered-To: apmail-activemq-users-archive@activemq.apache.org Received: (qmail 5730 invoked by uid 500); 12 Aug 2019 13:41:57 -0000 Mailing-List: contact users-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@activemq.apache.org Delivered-To: mailing list users@activemq.apache.org Received: (qmail 5714 invoked by uid 99); 12 Aug 2019 13:41:56 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Aug 2019 13:41:56 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 11CFE1829D5 for ; Mon, 12 Aug 2019 13:41:56 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.954 X-Spam-Level: * X-Spam-Status: No, score=1.954 tagged_above=-999 required=6.31 tests=[DKIM_ADSP_CUSTOM_MED=0.001, NML_ADSP_CUSTOM_MED=1.2, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_NEUTRAL=0.652, URIBL_BLOCKED=0.001, URI_HEX=0.1, URI_TRY_3LD=0.001] autolearn=disabled Received: from mx1-he-de.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id k7uYm9f-jMkb for ; Mon, 12 Aug 2019 13:41:54 +0000 (UTC) Received-SPF: Neutral (mailfrom) identity=mailfrom; client-ip=199.38.86.66; helo=n4.nabble.com; envelope-from=charon.guillaume@yahoo.fr; receiver= Received: from n4.nabble.com (n4.nabble.com [199.38.86.66]) by mx1-he-de.apache.org (ASF Mail Server at mx1-he-de.apache.org) with ESMTP id F08697DD30 for ; Mon, 12 Aug 2019 13:41:53 +0000 (UTC) Received: from n4.nabble.com (localhost [127.0.0.1]) by n4.nabble.com (Postfix) with ESMTP id 14BB4AD24DEE for ; Mon, 12 Aug 2019 08:41:48 -0500 (CDT) Date: Mon, 12 Aug 2019 08:41:48 -0500 (CDT) From: ghuillaume To: users@activemq.apache.org Message-ID: <1565617308083-0.post@n4.nabble.com> Subject: Poor performances with large number of consumers MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hello, I have a network of 2 non-persistent brokers which is dedicated to dispatch messages to 10000 different consumers using a single topic. Each consumer use a topic selector to filter messages. The messages are produced by hundreds of producers : multiple Java/SpringBoot instances with a CachedConnexionFactory. I have other queues and topics but it's not the point here. My problem is that the more I have consumers on my topics, the more I have latency to deliver messages. I've designed a JMeter test suite to produce messages and measure performances easily, below the results : - With no consumer on the topic : up to 10k msg/second can be delivered with good performances (max/average time in ms reported by Jmeter : 400/0,8). JMeter is not able to produce more messages, I can't measure performances with higher throughput. - With 10K consumers : from 1k msg/second, poor performances (max/average time in ms reported by Jmeter : 30000/31). I have good performances with lower throughput All my clients (producers and consumers) are connected on the 2 brokers using failover transport reference, so I don't know on which broker the messages will be produced to and I don't know on which broker a message need to be forwarded to be consumed. You will find my config in attachment, and below some details about it : - conduitSubscriptions=false on networkConnector : because I use topic selectors - advisorySupport=false : because if true it doubles the number of consumers on my topic in a 2 nodes network scenario (i.e. : if I have 10K consumers, I will have 10K on each instance instead of 5K per instance), which impact negatively the performances - All my topics and queues are "staticallyIncludedDestinations" : because dynamic require advisorySupport - MaxPageSize=20000 on the topic policyEntry : it seems to be efficient. I think I need a large number to be able to evaluate a large amout of topic selector at the same time - constantPendingMessageLimitStrategy=20 on the topic because I have a lot of consumers (20 x 10000 = 2000000 pending messages) - The two brokers declare the same transport connector so it results in a duplex bridge between them. I'm using ActiveMQ 5.15.3 activemq.xml -- Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User-f2341805.html