couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <>
Subject Re: Practical storage limit
Date Sun, 13 Jul 2008 18:51:26 GMT

On Jul 13, 2008, at 20:25, Peter Eddy wrote:

> On Sun, Jul 13, 2008 at 12:46 PM, Jan Lehnardt <> wrote:
>> To get on write view update semantics, you can write a little daemon
>> script that runs alongside CouchDB and is specified in couch.ini
>> as DbUpdateNotificationProcesses. This deamon gets sent a
>> notification each time the database is changed and could in turn
>> trigger a view update every N document inserts or every Y seconds,
>> whichever occurs first. The reason not to integrate each doc as
>> it comes in is that it is horribly inefficient and CouchDB is  
>> designed
>> to do view index updates very fast, so batching is a good idea.
> Thanks for this information, Jan. I think this solution is a lot
> better than trying to figure out when it's necessary to update views
> in application code. However I'd still argue that people will want
> this as their default behavior most of the time, and I wonder if it
> wouldn't be more efficient to just build the daemon functionality into
> the server, maybe with tunable N and Y values.

I have no number to back it up, but in a usual web-app reads
outnumber writes by far and the incremental index update will
be fast enough. Yes you can always contrive counter examples,
but they can use the proposed solution.

I don't think that an erlang module to handle this would be that
much more efficient than an external daemon. After all, it
is only a few bytes exchanged. But as outlined in my last
paragraph, it makes sense to bundle such a system with
CouchDB but we haven't gotten to it yet. (This is your cue
to jump in *hint* :)


> - Peter
>> To get a list of all views in a database, you can do a
>> GET /db/_all_docs?startkey=_design/&endkey=_design/ZZZZ
>> (we will have a /db/_all_design_docs view to make the ZZZZ-hack
>> go away).
>> That should solve your problem.
>> Yes, such a daemon should be shipped with CouchDB, but we
>> haven't got around to work on the deployment infrastructure yet.
>> Any contributions to this are very welcome. I think the developer's
>> choice of language for helper scripts is Python, but any will do,
>> whatever suits you best.
>> Cheers
>> Jan
>> --
>> The FaQ entry is at the bottom of this page.
>> BTW: Is there a FaQ module or whatever for MoinMoin? it
>> would be nice to get a MediaWiki-like table of contents at
>> the top. Noah?

View raw message