incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eric casteleijn <eric.castele...@canonical.com>
Subject Re: Document update notifications
Date Mon, 01 Jun 2009 14:19:44 GMT
Chris Anderson wrote:
> The next step on the roadmap is to give the ability to filter the
> comet notifications with a JavaScript function. This code should not
> be too hard to write but I doubt I'll have time to get to it this
> week. There are a few other people who are capable of adding the
> function, if any of them add the capability I'll be happy to review
> and apply the patch. Otherwise it should just be a matter of time
> before the comet feature is complete.
> 
> There are other parts of the story we could add but haven't. For
> instance, it should be possible to do a long poll on a particular
> document URL if you are waiting for it to be modified. Also,
> replication should be able to run against the _changes API instead of
> _all_docs_by_seq, although I'm not sure that's crucial for your use
> case.
> 
> I hope that give enough clarity for you.

That is very good news! I did not know that the _changes view was 
already in trunk. I'll get it up and running and see if I can change our 
code to start listening to it.

When you talk about filtering, does that mean taking the output it 
currently gives and suppressing lines (which contain seq, doc_id and 
list of revisions, if I understand correctly) or to allow emitting other 
fields, or depending on the values of other fields as well?

The use case I am thinking of again is to be able to distinguish between 
creation, modification and deletion of documents. Perhaps the first 
could be done by inspecting revision numbers, although that may be 
relying on internals too much, and I don't know what could be done to 
detect whether a document was deleted.

The alternative I have seen suggested a lot, to have separate documents 
that record deletion/creation events is hard to implement when the code 
  accessing the database is not the same on all nodes (i.e. you have no 
control over some nodes, but would still like to have an idea of when 
documents that were created there were first seen/deleted by the node 
you do control.)

Also it concerns me that the _changes feed is now per database, rather 
than per server, (although there is 
https://issues.apache.org/jira/browse/COUCHDB-360 which implies a global 
_changes feed, but it seems that that in turn will only include the 
database name.)

Having to listen to one feed per database seems like it could take up a 
lot of resources if one has a very large number of databases, which I 
imagine we will.

Having to constantly query the databases for more information after each 
update also seems like it would become a performance burden very soon. 
This is one of the main reasons (the other being couchdb running 
asynchronous, and hence hard to test, processes) that I am looking 
forward to seeing the end of the update triggers as they are.

Ideally, I would like to have one _changes feed per server, with 
configurable output, so that any external processing that needs to 
happen can happen by one process per server, that will not have to query 
into the database after every received update.

-- 
- eric casteleijn
http://www.canonical.com

Mime
View raw message