Return-Path: X-Original-To: apmail-activemq-users-archive@www.apache.org Delivered-To: apmail-activemq-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 56447646C for ; Thu, 16 Jun 2011 12:45:58 +0000 (UTC) Received: (qmail 86528 invoked by uid 500); 16 Jun 2011 12:45:57 -0000 Delivered-To: apmail-activemq-users-archive@activemq.apache.org Received: (qmail 86495 invoked by uid 500); 16 Jun 2011 12:45:57 -0000 Mailing-List: contact users-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@activemq.apache.org Delivered-To: mailing list users@activemq.apache.org Received: (qmail 86487 invoked by uid 99); 16 Jun 2011 12:45:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Jun 2011 12:45:57 +0000 X-ASF-Spam-Status: No, hits=2.3 required=5.0 tests=SPF_SOFTFAIL,URI_HEX X-Spam-Check-By: apache.org Received-SPF: softfail (nike.apache.org: transitioning domain of ozan.seymen@tdpg.com does not designate 216.139.236.26 as permitted sender) Received: from [216.139.236.26] (HELO sam.nabble.com) (216.139.236.26) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 16 Jun 2011 12:45:51 +0000 Received: from joe.nabble.com ([192.168.236.139]) by sam.nabble.com with esmtp (Exim 4.72) (envelope-from ) id 1QXBww-0004VI-7y for users@activemq.apache.org; Thu, 16 Jun 2011 05:45:30 -0700 Date: Thu, 16 Jun 2011 05:45:30 -0700 (PDT) From: oseymen To: users@activemq.apache.org Message-ID: <1308228330231-3602343.post@n4.nabble.com> In-Reply-To: <1307815529968-3590852.post@n4.nabble.com> References: <1307720064197-3588714.post@n4.nabble.com> <1307815529968-3590852.post@n4.nabble.com> Subject: Re: Message loss in network of brokers - transactional send MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Here is some more info: I enabled debug logging and can see where the problem is (however I still don't know what the solution is). I ran my test again and send 5000 messages to one broker which is configured to store-and-forward the message to broker2. Two brokers are connected via network of brokers. After killing brokers randomly, I ended up 4997 messages in broker2. I wrote a consumer that will consume all messages from broker2 and create a spreadsheet with all properties. Using this spreadsheet I can pinpoint which messages have failed using sequential ids. One of the messages that has failed is #3751. Looking at the debug log I can see: (line 976) bridging (broker1 -> broker2) messageId = ID:HAM-NB-073-56843-634438260819794239-1:0:1:1:3748 (line 979) bridging (broker1 -> broker2) messageId = ID:HAM-NB-073-56843-634438260819794239-1:0:1:1:3749 (line 983) bridging (broker1 -> broker2) messageId = ID:HAM-NB-073-56843-634438260819794239-1:0:1:1:3750 Then broker1 is killed. When it comes back, it says: (line 1311) last stored sequence id set: 3751 (line 1312) suppressing duplicate message send [ID:HAM-NB-073-56843-634438260819794239-1:0:1:1:3751] with producerSequenceId [3751] less than last stored: 3751 There is not message sent information in the log for 3751!!! I've attached the full log for your perusal. I'd appreciate any help to solve this problem. This test was done with apache-activemq-5.4.2-fuse-02-00. http://activemq.2283324.n4.nabble.com/file/n3602343/activemq.log.4 activemq.log.4 -- View this message in context: http://activemq.2283324.n4.nabble.com/Message-loss-in-network-of-brokers-transactional-send-tp3588714p3602343.html Sent from the ActiveMQ - User mailing list archive at Nabble.com.