Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 81B84200D2D for ; Fri, 27 Oct 2017 14:41:35 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 804E41609E9; Fri, 27 Oct 2017 12:41:35 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C3822160BDC for ; Fri, 27 Oct 2017 14:41:34 +0200 (CEST) Received: (qmail 43267 invoked by uid 500); 27 Oct 2017 12:41:33 -0000 Mailing-List: contact users-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@activemq.apache.org Delivered-To: mailing list users@activemq.apache.org Received: (qmail 43087 invoked by uid 99); 27 Oct 2017 12:41:33 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 27 Oct 2017 12:41:33 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id B6A9E180157 for ; Fri, 27 Oct 2017 12:41:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 3.267 X-Spam-Level: *** X-Spam-Status: No, score=3.267 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, KAM_BADIPHTTP=2, KAM_SHORT=0.001, NORMAL_HTTP_TO_IP=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-2.8, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, URI_HEX=1.313, URI_TRY_3LD=0.001, WEIRD_PORT=0.001] autolearn=disabled Authentication-Results: spamd3-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id u7_GjHq5H5dt for ; Fri, 27 Oct 2017 12:41:31 +0000 (UTC) Received: from mail-io0-f180.google.com (mail-io0-f180.google.com [209.85.223.180]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id CB3025FB32 for ; Fri, 27 Oct 2017 12:41:30 +0000 (UTC) Received: by mail-io0-f180.google.com with SMTP id b186so12413998iof.8 for ; Fri, 27 Oct 2017 05:41:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=EXv3BlS5o9h2rB958+mZkYC1joaKDclBuRIxyPN1odI=; b=eC2FdXWHPJ9QMe6xk+1ehUokoGVaDqp4AKDXoIyDwunZqYTT0FLhnMW0H152iFXRFr h4OmDEo8punY/B3YNq0Z1SW7bdjyznmMXFRkUWzisDAvS8m6sAq8zSmhYYIFRBmeIk6g kxQjhkgHSKnVlTVfv//IPa5/UaabH5fD7nBAS0MxIfxP1SgnCenqTE+bgw19U14CKHOu BtDQe+CQuJ251xKStmDrtR1XIHpsMTSfR9SorWfEb65Im0r8mkJhRGQGT6navhIHL4Xz qnsdQWbYL4w127cGgcpRAWrRRNJoYQ9N8QtHPy2T/WZOuu48/q613Nzn24TIlPYnAQDF NwWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=EXv3BlS5o9h2rB958+mZkYC1joaKDclBuRIxyPN1odI=; b=UV6IOJb7qRw+4hr+SElpLYMvnQIZZF2Pwm8zVpaHzVdMjaZdlQdOLFz/xt9nWuHNzg 0Mhd5dW26YOgdJMMH6YXOPz7khHZC7fXkJQyu2oZEzsukN0J5YCnc3xN+NzPD9khC185 jIF+avD0k0rQS8bDd50fLk/V4cfWLomlncBm5+rq2K08g0AZq0apLt5RPBIlgRPZPzQU 0AxNaXKQqDaA3RrZIQdC1qsBoxp1G4z0nhQQs8a8McxAnKzNF7s7bHcC8acgAoR5aFcs lGrjNpbI8BJaqyys5aBIRIPl3ISxrj59MniZLdzkJJkEhmR5hvJe//0xJJgjCSWCfmYn 25BA== X-Gm-Message-State: AMCzsaW8Jj4sANsrdjQo1X2mJpqxM8p48kFnRKipjPRMvHHRDntKmDuz DiF8sNWx7Qmc1Z5DDDROkFCDRfWcxjFvbN447c0= X-Google-Smtp-Source: ABhQp+SOOcYpfXamzpHvcHZHMmD7vgXkok+Nh6KtV+N3UZnbQOGBzRsfVQVM+KAGfQ2WejWp8w2KykURkLQtvjlvXDY= X-Received: by 10.107.3.193 with SMTP id e62mr438012ioi.88.1509108083783; Fri, 27 Oct 2017 05:41:23 -0700 (PDT) MIME-Version: 1.0 Sender: tbain98@gmail.com Received: by 10.2.102.30 with HTTP; Fri, 27 Oct 2017 05:41:23 -0700 (PDT) Received: by 10.2.102.30 with HTTP; Fri, 27 Oct 2017 05:41:23 -0700 (PDT) In-Reply-To: <1509092386548-0.post@n4.nabble.com> References: <1509092386548-0.post@n4.nabble.com> From: Tim Bain Date: Fri, 27 Oct 2017 06:41:23 -0600 X-Google-Sender-Auth: DvNDZ7FI_gZRgQHbtEzS8cReIOw Message-ID: Subject: Re: MQTT Subscriber gets disconnected frequently when there are large number of MQTT clients connected To: ActiveMQ Users Content-Type: multipart/alternative; boundary="001a113ecaee432937055c869bab" archived-at: Fri, 27 Oct 2017 12:41:35 -0000 --001a113ecaee432937055c869bab Content-Type: text/plain; charset="UTF-8" Is there anything in the broker's logs when this happens? I believe the log content you quoted was from the client, not the broker, and I wasn't sure if what you wrote meant that there was nothing in the broker's logs, so I want to make sure. Can you reproduce the problem in a non-production environment where the use of a debugger or a profiler or increasing the logging level of the broker would be an option? Those techniques are the ones that jump to mind for how to investigate further, starting with increasing the logging level. Tim On Oct 27, 2017 2:19 AM, "Shobhana" wrote: > Background : We use AMQ to exchange MQTT non-persistent messages between > our > server components and clients (Android/iOS devices). About 100-200 > messages/second get exchanged during peak hours. > > This is our MQTT transport connector configuration : > uri="mqtt+nio://0.0.0.0:1883?maximumConnections=200000&wireFormat. > maxInactivityDuration=180000&wireFormat.maxFrameSize= > 104857600&jms.useAsyncSend=true&jms.alwaysSessionAsync=false"/> > > When there are more than 10000 MQTT connections (and roughly about 30000 > topics), the MQTT subscribers get disconnected. Even publish operations > time > out (timeout is set to 60 seconds on the publisher and subscriber end). > After a few attempts, connection gets re-established. Once broker enters > this state, disconnects become more frequent and after sometime, the > reconnect also fails forever until the broker is restarted! > > Any suggestions on how to investigate why this happens? Since this is a > production environment, I can't debug/profile the broker. I have checked > memory usage and CPU usage .. both are well under control. Thread dumps > show > that there are no blocked threads. Log level is set to WARN only. Sometimes > I see following error in log files when this happens : > > > > Any clue about what is happening or how to investigate further? > > TIA, > Regards, > Shobhana > > > > -- > Sent from: http://activemq.2283324.n4.nabble.com/ActiveMQ-User- > f2341805.html > --001a113ecaee432937055c869bab--