incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Bartell <snbart...@gmail.com>
Subject Re: replication by view
Date Wed, 30 Jan 2013 18:06:02 GMT

On Jan 30, 2013, at 12:13 AM, Benoit Chesneau <bchesneau@gmail.com> wrote:

> On Wed, Jan 30, 2013 at 8:53 AM, Stephen Bartell <snbartell@gmail.com>wrote:
> 
>> Benoit,  I'm curious about a comment you made earlier (from Nathans
>> half-baked idea) and have a few questions to spawn from it:
>> 
>> ```
>> There is another solution though, replication using a view change and real
>> replication using a view like in rcouch [1]. With the validate_doc_read
>> function [2]  you can do that. No need for a background process.  Hopefully
>> this will be merged in couchdb. This have been tested on a relatively large
>> scale
>> ```
>> 
>> 1) If I understand the couch docs correctly, using views as replication
>> filters yields no performance benefit.  Its just like running plain jane
>> filters.  I figure thats why you are mentioning rcouch where the
>> replication source is indexed data.  I just want to clarify that I
>> understand this.
>> 
> 
> In rcouch the implementation is different this is a real view changes: each
> time a change is indexed in a view an event is sent. All changes are also
> indexed in a btree so you can use a since= parameter. It is working like
> changes on the document database but on an index.
> 
> Since it's a real view you can also query changes in a range
> (start_key/end_key ) or for a key.
> 

This seems like a huge improvement! Replicating with filters always bothered me because a
first time replication would need to traverse the entire database.  Not to mention the timeout
issues which could occur if the db is massive.  I just figured it was a necessary evil.

> 
>> 2) For each replicator filter, or view that is run, is a single couchjs
>> process spawned to handle evaling the javascript?
>> 
> 
> No. It is just getting changes from a the index. There is no couchjs
> process used except if you also want to filter changes in that view.

Right, but in the case of a javascript filter (not the uber-sweet view changes), is a single
couchjs process spawned per filter?

> 
> 
>> 
>> 3) What if we write our views in erlang and then use those as replication
>> filters?  Could we get around the extra overhead of couchjs processes this
>> way?
>> 
> 
> 
> Again see above; View changes is not about filtering. Though you can filter
> these changes with an erlang function.
> 
>> 
>> 4) Can filters be erlang, thus getting around extra couchjs process
>> overhead?
>> 
> 
> Yes but it won't be sandboxed.
> 
> 
> - benoƮt


Mime
View raw message