incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Lazy Fulltext Search
Date Fri, 11 Apr 2008 11:24:50 GMT

On Apr 11, 2008, at 13:11, Noah Slater wrote:
> On Fri, Apr 11, 2008 at 12:37:30PM +0200, Jan Lehnardt wrote:
>> The associated benefit is that you delay the costs of generation of
>> indexes until you actually need them.
>
> If you're generating indexes JIT, you can't really count them as  
> indexes any
> more, you're essentially doing regular non-indexed searching.
>
> I would have thought that for a database the trade-off you want to  
> make is one
> where you sacrifice time/resources in bulk so that queries are  
> lighting fast.
>
> If you move the indexing to query time you still have to expend  
> exactly the same
> time/resources as before and you have slowed down your query  
> response time
> significantly. For large collections of documents, indexing could  
> easily take
> hours to complete.
>
>>> My understanding is that the KEY element of CouchDB Wiews is that  
>>> they are
>>> generated in advance, and incrementally, before you use them.
>>
>> And why not use the same principle fot fulltext indexes?
>
> I thought this was the original plan for the full text search, that  
> the index
> was built in advance and incrementally before you use it. It sounds  
> to me like
> you're suggesting a departure away from this.
>
> Maybe I am getting confused.

Yeah I don't know what is going wrong here. My words might be not  
clear enough. Sorry. Views are built JIT as well, with intermediate  
results cached, so that only latest changes need to be indexed. I  
propose the exactly same thing for fulltext searching. The same trade- 
offs apply and the same drawbacks as well. What I don't understand now  
is why you say it is good for views and bad for fulltext searching.  
The only architectural change I propose is that the indexer is  
triggered by the searcher on demand instead of CouchDB's update  
notification mechanism.

Still unclear?

Cheers
Jan
--

Mime
View raw message