From users-return-49794-archive-asf-public=cust-asf.ponee.io@activemq.apache.org Thu Apr 5 13:57:13 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id E5DD918063B for ; Thu, 5 Apr 2018 13:57:12 +0200 (CEST) Received: (qmail 81260 invoked by uid 500); 5 Apr 2018 11:57:11 -0000 Mailing-List: contact users-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@activemq.apache.org Delivered-To: mailing list users@activemq.apache.org Received: (qmail 81233 invoked by uid 99); 5 Apr 2018 11:57:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Apr 2018 11:57:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 60E53C00CA for ; Thu, 5 Apr 2018 11:57:10 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.249 X-Spam-Level: ** X-Spam-Status: No, score=2.249 tagged_above=-999 required=6.31 tests=[FREEMAIL_ENVFROM_END_DIGIT=0.25, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id 6PwEtiAmzqXD for ; Thu, 5 Apr 2018 11:57:08 +0000 (UTC) Received: from mail-wr0-f171.google.com (mail-wr0-f171.google.com [209.85.128.171]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 23CA55F33E for ; Thu, 5 Apr 2018 11:57:08 +0000 (UTC) Received: by mail-wr0-f171.google.com with SMTP id n2so24601115wrj.7 for ; Thu, 05 Apr 2018 04:57:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=kfUfsia6vWBLZvmN3GQKCy4E/x5yrqVNUIvlKiG80Xs=; b=KBWZ+PngshXfwg0SQFgE83LKqf1UfUX0rwGJmI8/cMzqgzximTEh0Y5VGC6sUkD8qb WX/lQnK0JvlqEmUPLCeFIMC2KTL/0iMnU7yABMrsoUBJxipUJzw/8m+7KBJMnvzsFl5Q mPUGvTGN85zZuafMJGb2DJI2hsc6xUlcD9O0iQP9HDo+QPRxjhnP9puHZFacwwhI1c+t gTI8PA8/foyJqITHFb7EdyVVWHsouEXckAVrD8zBNgHbDvzdNBEjwlwCrMi2tB//0Pnk gWUHkyRLfBNrVzLWsxRl+J0fewdp52F23Q7Ei0XQ7ZUetGcl9i47Y9D9zNodljS46/BP B4Ng== X-Gm-Message-State: AElRT7F0xIlAS/62i7E7H1FYxPU6/XeKuUo5Lczz4xlmhAoPZeKP6iwd cafhupFutzcTvOZdfTAvOjrrnuKK6vj9W3AdigM= X-Google-Smtp-Source: AIpwx49TwGacE2AJFRcCvWpsYGvY6rJ2H4eKY9B10sWUKOSslETJM4ErPrONpegos+xwIRqAc0F/7u92bpHIlrM5zE4= X-Received: by 10.223.136.13 with SMTP id d13mr15205274wrd.271.1522929421836; Thu, 05 Apr 2018 04:57:01 -0700 (PDT) MIME-Version: 1.0 References: <1522907503362-0.post@n4.nabble.com> In-Reply-To: <1522907503362-0.post@n4.nabble.com> From: Tim Bain Date: Thu, 05 Apr 2018 11:56:51 +0000 Message-ID: Subject: Re: Replicated Message Store for ActiveMQ To: ActiveMQ Users Content-Type: multipart/alternative; boundary="001a1146107c34e808056918a3ce" --001a1146107c34e808056918a3ce Content-Type: text/plain; charset="UTF-8" On Wed, Apr 4, 2018, 11:51 PM SubashKunjupillai wrote: > Hi, > > We have been using ActiveMQ 5.x (upgraded to 5.14 last year) for our > product > which is in production for 3years. We have been facing stability issues > with > replicated LevelDB store(it was deprecated by community after we went live > with LevelDB, we have stuck to it as we accomplished HA through replicated > message store which is not available in KahaDB). Now we have reached a > threshold point where we can't withstand any further LevelDB corrupt and > lose the messages store over there. > > We are looking for the possible way forward to solve this and it would be > great if the community can help us on identifying the right solution. > > Just a high level picture on our architecture. We have 3 brokers running in > three machines with N number of producers and N number of consumers which > can be scaled independently. 3 instance of brokers are maintained to > provide > HA using replicated LevelDB, so that even at 2 node failure, message queue > is available with zero loss of messages. > > 1) Replace LevelDB with KahaDB and use Hadoop NFS (Faster than traditional > NFS provided by hadoop distributor MapR) to persist KahaDB data. So that > even if two of the broker goes down, the other broker can work on the data > available through Hadoop NFS. I guess it would same as Shared File Storage, > so this should work. Please confirm. > I'd be very skeptical about whether a Hadoop-based NFS implementation provides exclusive locks that are compatible with ActiveMQ's needs. I strongly recommend you test that thoroughly before committing to this path. NFS mount settings are the subject of several currently-active threads on this mailing list, so you can see the settings some other people have used. I'd also be very skeptical about whether a Hadoop-based NFS implementation is fast enough to support constant read-write activity on a small set of files, though it's not something I've ever researched personally so I may be overly pessimistic here. 2) Replace ActiveMQ 5.x with ActiveMQ Artemis. I can understand from the > Artemis documents that the replicated message store option is available. If > Artemis is chosen, we are speculating about the code changes and efforts > required to adapt the same when it will be release as ActiveMQ 6.x. Please > en light me on these. > I'd be surprised if the Artemis codebase changes at all when it becomes 6.x, so I don't think you have to worry about any effort to change your code later just because you start using Artemis now (before it's 6.x). Moving to Artemis seems like the best path forward for what you've described as your needs. Tim > --001a1146107c34e808056918a3ce--