From users-return-16125-apmail-jackrabbit-users-archive=jackrabbit.apache.org@jackrabbit.apache.org Tue Sep 14 11:15:47 2010 Return-Path: Delivered-To: apmail-jackrabbit-users-archive@minotaur.apache.org Received: (qmail 68360 invoked from network); 14 Sep 2010 11:15:47 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 14 Sep 2010 11:15:47 -0000 Received: (qmail 59977 invoked by uid 500); 14 Sep 2010 11:15:47 -0000 Delivered-To: apmail-jackrabbit-users-archive@jackrabbit.apache.org Received: (qmail 59866 invoked by uid 500); 14 Sep 2010 11:15:44 -0000 Mailing-List: contact users-help@jackrabbit.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: users@jackrabbit.apache.org Delivered-To: mailing list users@jackrabbit.apache.org Received: (qmail 59852 invoked by uid 99); 14 Sep 2010 11:15:43 -0000 Received: from Unknown (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Sep 2010 11:15:43 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=10.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of a.schrijvers@1hippo.com designates 64.18.2.179 as permitted sender) Received: from [64.18.2.179] (HELO exprod7og113.obsmtp.com) (64.18.2.179) by apache.org (qpsmtpd/0.29) with SMTP; Tue, 14 Sep 2010 11:15:20 +0000 Received: from source ([209.85.215.45]) by exprod7ob113.postini.com ([64.18.6.12]) with SMTP ID DSNKTI9ZMq0+geiFVEVsZWRXopzOI9ezpbva@postini.com; Tue, 14 Sep 2010 04:14:59 PDT Received: by ewy27 with SMTP id 27so3499238ewy.4 for ; Tue, 14 Sep 2010 04:14:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.213.113.206 with SMTP id b14mr2699246ebq.33.1284462897784; Tue, 14 Sep 2010 04:14:57 -0700 (PDT) Received: by 10.213.104.146 with HTTP; Tue, 14 Sep 2010 04:14:57 -0700 (PDT) In-Reply-To: <7063C839-29EB-46F2-9A6C-5F2156FAEFA7@tfd.co.uk> References: <7063C839-29EB-46F2-9A6C-5F2156FAEFA7@tfd.co.uk> Date: Tue, 14 Sep 2010 13:14:57 +0200 Message-ID: Subject: Re: Adding new nodes to a cluster From: Ard Schrijvers To: users@jackrabbit.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org On Tue, Sep 14, 2010 at 11:55 AM, Ian Boston wrote: > > On 14 Sep 2010, at 19:41, Ard Schrijvers wrote: >>> >>> 1. Take a complete snapshot (disk image) of one of the live nodes - >>> including the Lucene index >> >> Although I am not to familiar with the clustered setup (others at my >> company are), I know that this is not possible unfortunately. The >> problem is that the most recent Lucene index is an in-memory one. You >> cannot get correct snapshots from the index. It is something I'd love >> to get improved in some time in Jackrabbit. > > OMG, Ard, =A0no, really, did that come in with Jackrabbit 2 or was it als= o the case in JR1.x ? This was part of JR1.x. Back in the days, when re-opening an index reader in Lucene was not possible, there was a very good reason for doing so. That's why (I wasn't involved yet though, so I am a little guessing but qute confident), jr originally build a very nice state of the art multi-index solution, with merging between indexes just like segments. This combined with read only indexreaders, internal memory bitsets keeping track of deleted lucene docs etc etc, was I think ahead of Lucene. However, Lucene now does pretty much all Jackrabbit needed to solve internally within a single lucene index. I did some testing, and I see no reason anymore to keep a multi-index. Our needs are very comparable to Hibernate, however, they had the luck they started with Lucene when this was already in place. Current jackrabbit lucene architecture also doesn't fit something like infinispan LuceneDirectory (it needs a reopen() on every call), which would be a very nice thing to be able to use. Anyway, tons of ideas I have, all I need is some (much) time :-((( > Is there anyway to get it pushed to disk when a snapshot is required. I don't think so. Lucene has these day a snapshot policy you can make use of. Didn't look at it, but it would be one of the things I'd certainly like to have (just as rebuilding an entire index in the background to replace an existing one when it is finished) > We =A0(also) have a requirement to run in a cluster and we cant a) hold a= journal going back to 0, that would soon be impractical and b) cant take n= odes offline to snapshot > > Well we could do (b) but it would be very bad politically. I would have to ask how our sysadmins do it...I unfortunately do not know this, but I know it should be easier then it currently is Regards Ard > > Ian > >