Mailing-List: contact oak-dev-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: oak-dev@jackrabbit.apache.org
Received-SPF: pass (nike.apache.org: domain of chetan.mehrotra@gmail.com
 designates 209.85.214.175 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <a749aa32d2c746c0993c889313455a89@BY2PR02MB266.namprd02.prod.outlook.com>
References: 
 <CAHCW-mmz=hohW4oDSbd_Y=REM7hqPC9sLUQ7eC8ymAYw1F2Fow@mail.gmail.com>
	<CAOFYJNatseNga7qLLbXK5LCNLJ4Mm6EamuWBXvAoczfdSfD1yA@mail.gmail.com>
	<a749aa32d2c746c0993c889313455a89@BY2PR02MB266.namprd02.prod.outlook.com>
Date: Wed, 9 Apr 2014 14:13:33 +0530
Message-ID: 
 <CAHCW-mk3+9zXz3Wx6j4eFyXCVBKigqgrN7yo2NBU8PR4NqMU4Q@mail.gmail.com>
Subject: Re: Slow full text query performance and Lucene Index handling in Oak
From: Chetan Mehrotra <chetan.mehrotra@gmail.com>
To: oak-dev@jackrabbit.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Wed, Apr 9, 2014 at 12:25 PM, Marcel Reutegger <mreutegg@adobe.com> wrote:
>> Since the Lucene index is in any case updated asynchronously, it
>> should be fine for us to ignore the base NodeState of the current
>> session and instead use an IndexSearcher based on the last state as
>> updated by the async indexer. This would allow us to reuse the
>> IndexSearcher over multiple queries.
>
> I was also wondering if it makes sense to share it across multiple
> sessions performing a query to reduce the number of index readers
> that may be open at the same time. however, this will likely also
> reduce concurrency because we synchronize access to a single
> session.

I tried with one approach where I used a custom SerahcerManager based
on Lucene SearcherManager. It obtains the root NodeState directly from
NodeStore. As NodeStore can be accessed concurrently it should not
have any impact on session concurrency

With this change there is a slight improvement

Oak-Tar                            1      39      40      40      44
   64    1459
Oak-Tar(Shared)                    1      32      33      34      36
   61    1738

So did not gave much boost (at least with approach taken). As I do not
have much understanding of Lucene internal can someone review the
approach taken and see if there are some major issues with it


Chetan Mehrotra
[1] https://issues.apache.org/jira/secure/attachment/12639366/OAK-1702-shared-indexer.patch
[2] https://lucene.apache.org/core/3_6_0/api/all/org/apache/lucene/search/SearcherManager.html