couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: View Filter
Date Wed, 13 May 2009 21:31:07 GMT
On Wed, May 13, 2009 at 5:26 PM, kowsik <kowsik@gmail.com> wrote:
> We had a thread a month or so ago about view server optimization where
> this came up. By having document 'classes', it's possible to have
> design views that only get applied to certain documents.
>
> While I like this, I can immediately see someone asking for AND
> instead of the OR (implemented below) in the filter. The next
> generalization will be not just top level attributes, but nested ones
> too.
>
> So the optimization really is that when there are multiple view
> functions in a single design document that all apply to the same
> document class, we want to reject early and not have to invoke the map
> function.
>
> One possible alternative is to have the filter implemented in the view
> server, but invoked once for each design document (potentially with
> multiple views). This allows the user to control what the filter
> predicate looks like and can implement any scheme that s/he likes.
>
> Thoughts?
>

My guess is that most of the speedup of filtering comes from avoiding
the serialization/deserialization dance, not the actual application of
JavaScript methods. Ie, once the view server is computing the filter,
its probably little to no savings.

HTH,
Paul

> K.
>
> On Wed, May 13, 2009 at 7:41 AM, Zachary Zolton
> <zachary.zolton@gmail.com> wrote:
>> So, this sounds like a big win for those who like to store many
>> document types in the same database with a "type descriminator" field.
>>
>> It often takes me a a bit of thinking to decide whether or not to
>> store different document in the same database. So, I suppose a feature
>> like this would alleviate a possible negative side effect of that
>> choice, by reducing the time it takes to filter out the different
>> document types. Shooting from the hip, I'd say I like this feature!
>>
>> –Zach
>>
>> On Wed, May 13, 2009 at 9:08 AM, Jan Lehnardt <jan@apache.org> wrote:
>>> Hi,
>>>
>>> I made views faster! :)
>>>
>>> I wrote a patch* that introduces the concept of a view filter.
>>> A new design doc option acts as a document filter and
>>> prevents a doc from getting serialized and sent to the view
>>> server. This is useful to avoid unnecessary computation
>>> when using views that use the `if(doc.type == "foo") {…`
>>> pattern.
>>>
>>> *
>>> http://github.com/janl/couchdb/commit/a47a4831db74e3e0400c6faaaa29984e10ac861c
>>>
>>> The filter works like this:
>>> {
>>>  _id:"_design/test_foo",
>>>  language: "javascript",
>>>  options: {
>>>    filter: [{type: "foo"}, {type: "bar"}] // oh hey, proplists in JSON! :)
>>>  },
>>>  views: {
>>>    all_docs: { // really, only all foo and bar docs
>>>      map: "function(doc) { emit(doc.integer, null); }"
>>>    }
>>>  }
>>> };
>>>
>>> If *any* of the `{field, value}`** objects match a *top level* field and
>>> value in a document, it gets sent to the view server.
>>>
>>> ** Yeah, `field` is not hardcoded to `type`, so if you use `class:"foo"`,
>>> this patch works for you.
>>>
>>> A few notes:
>>>
>>>  - It would be nice if we could extend this so…
>>>   No — I don't like to add any more bells an whistles to this as
>>>   eventually this will lead to pure Erlang views which we want
>>>   to get anyway.
>>>
>>>  - In the light of other view server improvements, this might prove
>>>   to gain only marginal speed.
>>>
>>>  - Can we have a filter per view, not a filter per design doc? — Not
>>>   without major reworking of the view server and with losing other
>>>   optimisations.
>>>
>>> I don't think I should just go and commit this without discussion. In
>>> fact, I'd opt to only include this patch if there's demand. I'm happy
>>> to maintain the patch outside of CouchDB for those who need that
>>> speedup.
>>>
>>> Cheers
>>> Jan
>>> --
>>>
>>>
>>
>

Mime
View raw message