Mailing-List: contact users-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@jackrabbit.apache.org
Received-SPF: pass (nike.apache.org: domain of d.g.van.der.laan@rug.nl
 designates 129.125.60.6 as permitted sender)
Message-ID: <4CA315F2.8070706@rug.nl>
Date: Wed, 29 Sep 2010 12:33:22 +0200
From: Dennis van der Laan <d.g.van.der.laan@rug.nl>
Organization: Rijksuniversiteit Groningen / Centrum voor Informatietechnologie
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US;
 rv:1.9.2.9) Gecko/20100915 Lightning/1.0b2 Thunderbird/3.1.4
MIME-Version: 1.0
To: users@jackrabbit.apache.org
Subject: Re: Lucene consistency in clustered environment
References: <4CA1CD76.9010103@rug.nl>
 <33BB0FEA-96AD-4EA0-B5EE-4AE0C212B693@tfd.co.uk>
In-Reply-To: <33BB0FEA-96AD-4EA0-B5EE-4AE0C212B693@tfd.co.uk>
Content-Type: multipart/mixed;
 boundary="------------090705060909050605040805"

--------------090705060909050605040805
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

 Thanks for the reply. We did setup all machines as a Jackrabbit cluster
from the start.  We don't have a complete journal starting from the
first operation ever done on the repository. One node is running the
janitor process, so if all current nodes are in sync, the old journal
entries are removed.

We currently rebuild the Lucene indexes by shutting down the
repositories one by one, removing the 'version' and 'workspaces' folder
from the repository root folder and then starting the repositories
again. All works well after that, but it is an ugly workaround.
Rebuilding the indexes currently takes about 2.5 hours per machine
(approx. 300.000 documents in the repository).

The only thing we see right now is that some documents don't seem to get
indexed on one machine, but do get indexed on another. From your reply I
understand that this should not be the case with Lucene, is it?

TIA,
Dennis

On 28-9-2010 14:19, Ian Boston wrote:
> Did you setup the machines as a Jackrabbit cluster as per [1] right from the start ?
> And do you have a complete Journal right from the first operation in the JCR (or local state snapshots)
>
> The local state of the Lucene index is dependent on every node in the cluster replaying every event from the Journal to ensure that they all contain the same content. The Journal is also used for sharedItem state invalidation which will ensure that stale items do not get into responses.
>
> HTH
> Ian
>
> 1 http://wiki.apache.org/jackrabbit/Clustering
>
> On 28 Sep 2010, at 12:11, Dennis van der Laan wrote:
>
>> Hi all,
>>
>> We are using Jackrabbit 1.6.1 in a production environment. It is
>> clustered across 6 machines, which all store the Lucene indexes on a
>> local disk. After using this setup for a couple of months, we are seeing
>> the Lucene indexes differ per machine. Not only the size of the indexes,
>> but some documents seem to be indexed on one machine, but not on another
>> machine. So doing a full-text search (xpath with a ' contains' clause)
>> will have different results depending on the machine the query is run on.
>> I am a complete Lucene novice, so my question is: is this
>> non-deterministic behaviour a characteristic of Lucene (and should we
>> rebuild all indexes on a regular basis to keep them in sync) or is
>> something going wrong here?
>>
>> Thanks for any help!
>> Beste regards,
>> Dennis
>>
>> -- 
>> Dennis van der Laan, MSc
>> Centre for Information Technology
>> University of Groningen
>>


-- 
Dennis van der Laan


--------------090705060909050605040805--