Return-Path: X-Original-To: apmail-activemq-users-archive@www.apache.org Delivered-To: apmail-activemq-users-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 385199B3A for ; Tue, 16 Dec 2014 15:40:25 +0000 (UTC) Received: (qmail 64769 invoked by uid 500); 16 Dec 2014 15:40:24 -0000 Delivered-To: apmail-activemq-users-archive@activemq.apache.org Received: (qmail 64720 invoked by uid 500); 16 Dec 2014 15:40:24 -0000 Mailing-List: contact users-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@activemq.apache.org Delivered-To: mailing list users@activemq.apache.org Received: (qmail 64678 invoked by uid 99); 16 Dec 2014 15:40:23 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Dec 2014 15:40:23 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of christiangrassi@gmail.com designates 209.85.212.176 as permitted sender) Received: from [209.85.212.176] (HELO mail-wi0-f176.google.com) (209.85.212.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 16 Dec 2014 15:39:57 +0000 Received: by mail-wi0-f176.google.com with SMTP id ex7so12650562wid.9 for ; Tue, 16 Dec 2014 07:39:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=9dTIymxbHI1a8QkXfKFUI+OPgR1vpYiJwExvenSWdEs=; b=N1u5SBMytrCgitTuAh8plsHa9pJBRkte2WIn7002Zo6gq0iiN5PUvQovQ8StbRVO5A CySC4li2v8aUzAze+MUPINZbuxHw6PgHOC8WdS+XUMm2F3qIv9YfHAGpHuIfOty40FXh f9v2HVp6BVtAc/n6LexcLRTnQ0h6x4Qpa0dle+QAKvrvaYXd3lgL8Qb57dax2KYrioIP 6pe917HOZi+uSv8vHmbgQmZPtCS/2hJIgeOCnW/uliln1Dn9tJsu97NJi1N6ED+Rrvet mPl9TRkf3pJfQ2AWoBbF+BPz999caR2sFnln7Xgh4m4Kt4LAx15Kz+Tc8gN+3WtFBiaJ ZHow== MIME-Version: 1.0 X-Received: by 10.194.81.38 with SMTP id w6mr62461430wjx.17.1418744351504; Tue, 16 Dec 2014 07:39:11 -0800 (PST) Received: by 10.217.79.72 with HTTP; Tue, 16 Dec 2014 07:39:11 -0800 (PST) Date: Tue, 16 Dec 2014 16:39:11 +0100 Message-ID: Subject: ActiveMQ 5.10 LevelDB Issues From: Christian Grassi To: users@activemq.apache.org Content-Type: multipart/alternative; boundary=047d7bf0c0861942fe050a5729a7 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bf0c0861942fe050a5729a7 Content-Type: text/plain; charset=UTF-8 I have a 3 node 5.10 cluster with replicated levelDB as persistent store. I'm Stresstesting the queue with around 5000 persistent msg/s. 1 Producer 2 Consumers I get some warning messages on the log: On Master: 2014-12-16 16:25:28,375 | INFO | Slave has disconnected: db6c9a23-7025-4384-b02f-dcda763113c3 | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | hawtdispatch-DEFAULT-3 2014-12-16 16:25:28,761 | WARN | Unexpected session error: java.io.FileNotFoundException: /queue/activemq/conf/activemq-data/00000027b8d097b0.log (No such file or directory) | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | hawtdispatch-DEFAULT-1 2014-12-16 16:25:28,761 | INFO | Slave has disconnected: 6a84579c-77fe-41eb-a728-1be472c12894 | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | hawtdispatch-DEFAULT-2 2014-12-16 16:25:29,603 | INFO | Slave has connected: db6c9a23-7025-4384-b02f-dcda763113c3 | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | hawtdispatch-DEFAULT-2 2014-12-16 16:25:30,034 | INFO | Slave has connected: 6a84579c-77fe-41eb-a728-1be472c12894 | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | hawtdispatch-DEFAULT-1 2014-12-16 16:25:31,360 | INFO | Slave has now caught up: 6a84579c-77fe-41eb-a728-1be472c12894 | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | hawtdispatch-DEFAULT-4 2014-12-16 16:25:32,546 | INFO | Slave has now caught up: db6c9a23-7025-4384-b02f-dcda763113c3 | org.apache.activemq.leveldb.replicated.MasterLevelDBStore | hawtdispatch-DEFAULT-2 On the slaves 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 2014-12-16 16:30:00,287 | WARN | No reader available for position: 27fd52c562, log_infos: {171757839210=LogInfo(/queue/activemq/conf/activemq-data/00000027fd90a36a.log,171757839210,0)} | org.apache.activemq.leveldb.RecordLog | Thread-16882 It looks to me that from time to time the connection to the master drops. Should I worry about it ? what No reader available means, is there data loss risk? It looks to me that the level db implementation is kind of fragile... Is somebody using it in production ? Regards Christian --047d7bf0c0861942fe050a5729a7--