Mailing-List: contact couchdb-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-dev@incubator.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Message-Id: <B4984BCD-ADB3-4818-AAEC-78132EC09E7D@apache.org>
From: Jan Lehnardt <jan@apache.org>
To: couchdb-dev@incubator.apache.org
In-Reply-To: <17413.193.3.142.123.1207896938.squirrel@www.widetrail.dk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Apple Message framework v919.2)
Subject: Re: Lazy Fulltext Search
Date: Fri, 11 Apr 2008 13:26:47 +0200
References: <D27A5EDE-DEC9-457F-8C85-E6653921B699@apache.org>
 <17413.193.3.142.123.1207896938.squirrel@www.widetrail.dk>


On Apr 11, 2008, at 08:55, S=F8ren Hilmer wrote:
> Hi Jan
>
> It certainly would simplify configuration, allthough the
> DbUpdateNotificationProcess setting ought to be retained as it is
> potentially usefull for other stuff than indexing (can you have more =20=

> than
> one of these, setup?)

No, the update searcher will stay! :-)


> I am also worried about responsetimes for searching, potentially the
> indexing can take considerable time. With the current approach =20
> indexing
> can be done off peak hours and only searching is done at prime time.

Right, if you want to be conservative with resources, you might want =20
togo
with my approach at the expense of possibly higher response times the
first time things are searched for (as it is with views). I just =20
wanted to make
available my idea that fulltext indexing could be modelled after how =20
views
work, in case this is useful for a specific scenario.

Cheers
Jan
--


>
>
> Have fun
>  S=F8ren
> --=20
> S=F8ren Hilmer, M.Sc., M.Crypt.
> wideTrail            Phone: +45 25481225
> Pilev=E6nget 41        Email: sh@widetrail.dk
> DK-8961  Alling=E5bro  Web: www.widetrail.dk
>
> On Thu, April 10, 2008 23:32, Jan Lehnardt wrote:
>> Heya,
>> while thinking more about the fulltext implementation, I began to
>> wonder why we don't model it after the view engine.
>>
>> At the moment, we have an Indexer waiting for update notifications =20=

>> and
>> polling CouchDB for changes and a separate mechanism to register a
>> fulltext query Searcher, that looks up things in the index.
>>
>> My proposed architectural change would be to trigger the Indexer from
>> the Searcher module when a request comes in, just like views work.
>> This would delay the creation of fulltext indexes until they are
>> actually needed.
>>
>> The possible drawback though is, that when building the fulltext =20
>> index
>> is rather slow, old-style pre-calculation might be more feasible. =20
>> View
>> deal with that by requiring frequent requests (possibly cron-ed).
>>
>> This is not a proposal or anything, just a thought I wanted to share
>> with those who work on fulltext integration.
>>
>> If you have any input on this, please let us know ;)
>>
>> Cheers
>> Jan
>> --
>>
>
>
>