Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: unknown (athena.apache.org: error in processing during lookup of
 jaap@brightin.nl)
Message-ID: <faaeb2d30809260454n7600d228lc52ee0e787859cf1@mail.gmail.com>
Date: Fri, 26 Sep 2008 13:54:06 +0200
From: "Jaap van der Plas" <jaap@brightin.nl>
To: couchdb-user@incubator.apache.org
Subject: Re: Multiple filters on a large data set
In-Reply-To: <1E0C812D-E968-4AEB-BA64-1DBDB2C1B099@apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <faaeb2d30809251125j179ba6e9m85f973f3714018c4@mail.gmail.com>
	 <1E0C812D-E968-4AEB-BA64-1DBDB2C1B099@apache.org>

Greetings Jan,

the goal is not so much to allow full-text search on the documents
fields, as to be able to retrieve records based any set of field names
with specific values (or perhaps ranges.) Basically, what the 'where'
clause can do in SQL.
I think this sort of a query can be expressed very well with a
JavaScript function, but as I understand it this would require either
temporary views or creating a view for every set of fields that is
being queried by the user. Would it be possible to define a view with
only a map function, and then specify the reduce function at query
time (through POST or otherwise)?


On Fri, Sep 26, 2008 at 12:31, Jan Lehnardt <jan@apache.org> wrote:
> Hello Jaap,
>
> On Sep 25, 2008, at 20:25 , Jaap van der Plas wrote:
>
>> We like to build a database with approx. 50.000 documents. Every
>> document has at least about 10 fields. We think CouchDB would be nice
>> solution because there's a big variety of documents (like different
>> fields).
>
> I'd agree on the assessment that CouchDB is a good fit here.
>
>
>> We intend to use this for an online catelog. We like end users to be
>> able to search with multiple fields (like filters). Any combination of
>> fields should be possible.
>>
>> 1. Is this possible to do this without creating temp_views for every
>> query?
>
> If you want to do fulltext search, I'd second Ayende's suggestion of
> using the external search API to let Lucene do this kind of indexing and
> searching.
>
> Views can get you there if you create a comprehensive index of all fields
> and all their combinations (possible, probably not nice) and allow for
> a keyword search. You could even split up your filter words and put single
> chars, tuples and so on into the index to fake fulltext search. This is
> possible
> but probably not the most elegant solution.
>
>
>> 2. If not, is using temp_views viable performance wise on this sort of
>> dataset.
>
> Using temp views is not wise unless you want to test view function
> definitions
> on rather little data. A production system should not rely on them. (This is
> again, a rule of thumb, but with a pretty big thumb!)
>
>
> Cheers
> Jan
> --
>