From couchdb-user-return-742-apmail-incubator-couchdb-user-archive=incubator.apache.org@incubator.apache.org Sun Jul 13 18:52:00 2008 Return-Path: Delivered-To: apmail-incubator-couchdb-user-archive@locus.apache.org Received: (qmail 61322 invoked from network); 13 Jul 2008 18:51:59 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 13 Jul 2008 18:51:59 -0000 Received: (qmail 45111 invoked by uid 500); 13 Jul 2008 18:51:59 -0000 Delivered-To: apmail-incubator-couchdb-user-archive@incubator.apache.org Received: (qmail 45074 invoked by uid 500); 13 Jul 2008 18:51:59 -0000 Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: couchdb-user@incubator.apache.org Delivered-To: mailing list couchdb-user@incubator.apache.org Received: (qmail 45063 invoked by uid 99); 13 Jul 2008 18:51:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Jul 2008 11:51:59 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [83.97.50.139] (HELO jan.prima.de) (83.97.50.139) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 13 Jul 2008 18:51:06 +0000 Received: from [10.0.2.3] (e179134130.adsl.alicedsl.de [::ffff:85.179.134.130]) (AUTH: LOGIN jan, SSL: TLSv1/SSLv3,128bits,AES128-SHA) by jan.prima.de with esmtp; Sun, 13 Jul 2008 18:51:27 +0000 Message-Id: <4632AD67-2D6E-4552-BC74-9CCDB0C2FB11@apache.org> From: Jan Lehnardt To: couchdb-user@incubator.apache.org In-Reply-To: <5aea26870807131125q56289a58r26a49d40a03addb0@mail.gmail.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v928.1) Subject: Re: Practical storage limit Date: Sun, 13 Jul 2008 20:51:26 +0200 References: <5aea26870807130905h12f47a37i426b39fd9f4c17d1@mail.gmail.com> <5aea26870807131125q56289a58r26a49d40a03addb0@mail.gmail.com> X-Mailer: Apple Mail (2.928.1) X-Virus-Checked: Checked by ClamAV on apache.org On Jul 13, 2008, at 20:25, Peter Eddy wrote: > On Sun, Jul 13, 2008 at 12:46 PM, Jan Lehnardt wrote: >> To get on write view update semantics, you can write a little daemon >> script that runs alongside CouchDB and is specified in couch.ini >> as DbUpdateNotificationProcesses. This deamon gets sent a >> notification each time the database is changed and could in turn >> trigger a view update every N document inserts or every Y seconds, >> whichever occurs first. The reason not to integrate each doc as >> it comes in is that it is horribly inefficient and CouchDB is >> designed >> to do view index updates very fast, so batching is a good idea. > > Thanks for this information, Jan. I think this solution is a lot > better than trying to figure out when it's necessary to update views > in application code. However I'd still argue that people will want > this as their default behavior most of the time, and I wonder if it > wouldn't be more efficient to just build the daemon functionality into > the server, maybe with tunable N and Y values. I have no number to back it up, but in a usual web-app reads outnumber writes by far and the incremental index update will be fast enough. Yes you can always contrive counter examples, but they can use the proposed solution. I don't think that an erlang module to handle this would be that much more efficient than an external daemon. After all, it is only a few bytes exchanged. But as outlined in my last paragraph, it makes sense to bundle such a system with CouchDB but we haven't gotten to it yet. (This is your cue to jump in *hint* :) Cheers Jan -- > > > - Peter > >> To get a list of all views in a database, you can do a >> GET /db/_all_docs?startkey=_design/&endkey=_design/ZZZZ >> (we will have a /db/_all_design_docs view to make the ZZZZ-hack >> go away). >> >> That should solve your problem. >> >> Yes, such a daemon should be shipped with CouchDB, but we >> haven't got around to work on the deployment infrastructure yet. >> Any contributions to this are very welcome. I think the developer's >> choice of language for helper scripts is Python, but any will do, >> whatever suits you best. >> >> Cheers >> Jan >> -- >> The FaQ entry is at the bottom of this page. >> http://wiki.apache.org/couchdb/FrequentlyAskedQuestions >> BTW: Is there a FaQ module or whatever for MoinMoin? it >> would be nice to get a MediaWiki-like table of contents at >> the top. Noah? >> >