incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kowsik <kow...@gmail.com>
Subject Re: View Filter
Date Wed, 13 May 2009 21:26:11 GMT
We had a thread a month or so ago about view server optimization where
this came up. By having document 'classes', it's possible to have
design views that only get applied to certain documents.

While I like this, I can immediately see someone asking for AND
instead of the OR (implemented below) in the filter. The next
generalization will be not just top level attributes, but nested ones
too.

So the optimization really is that when there are multiple view
functions in a single design document that all apply to the same
document class, we want to reject early and not have to invoke the map
function.

One possible alternative is to have the filter implemented in the view
server, but invoked once for each design document (potentially with
multiple views). This allows the user to control what the filter
predicate looks like and can implement any scheme that s/he likes.

Thoughts?

K.

On Wed, May 13, 2009 at 7:41 AM, Zachary Zolton
<zachary.zolton@gmail.com> wrote:
> So, this sounds like a big win for those who like to store many
> document types in the same database with a "type descriminator" field.
>
> It often takes me a a bit of thinking to decide whether or not to
> store different document in the same database. So, I suppose a feature
> like this would alleviate a possible negative side effect of that
> choice, by reducing the time it takes to filter out the different
> document types. Shooting from the hip, I'd say I like this feature!
>
> –Zach
>
> On Wed, May 13, 2009 at 9:08 AM, Jan Lehnardt <jan@apache.org> wrote:
>> Hi,
>>
>> I made views faster! :)
>>
>> I wrote a patch* that introduces the concept of a view filter.
>> A new design doc option acts as a document filter and
>> prevents a doc from getting serialized and sent to the view
>> server. This is useful to avoid unnecessary computation
>> when using views that use the `if(doc.type == "foo") {…`
>> pattern.
>>
>> *
>> http://github.com/janl/couchdb/commit/a47a4831db74e3e0400c6faaaa29984e10ac861c
>>
>> The filter works like this:
>> {
>>  _id:"_design/test_foo",
>>  language: "javascript",
>>  options: {
>>    filter: [{type: "foo"}, {type: "bar"}] // oh hey, proplists in JSON! :)
>>  },
>>  views: {
>>    all_docs: { // really, only all foo and bar docs
>>      map: "function(doc) { emit(doc.integer, null); }"
>>    }
>>  }
>> };
>>
>> If *any* of the `{field, value}`** objects match a *top level* field and
>> value in a document, it gets sent to the view server.
>>
>> ** Yeah, `field` is not hardcoded to `type`, so if you use `class:"foo"`,
>> this patch works for you.
>>
>> A few notes:
>>
>>  - It would be nice if we could extend this so…
>>   No — I don't like to add any more bells an whistles to this as
>>   eventually this will lead to pure Erlang views which we want
>>   to get anyway.
>>
>>  - In the light of other view server improvements, this might prove
>>   to gain only marginal speed.
>>
>>  - Can we have a filter per view, not a filter per design doc? — Not
>>   without major reworking of the view server and with losing other
>>   optimisations.
>>
>> I don't think I should just go and commit this without discussion. In
>> fact, I'd opt to only include this patch if there's demand. I'm happy
>> to maintain the patch outside of CouchDB for those who need that
>> speedup.
>>
>> Cheers
>> Jan
>> --
>>
>>
>

Mime
View raw message