From user-return-22491-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Wed Oct 10 12:19:44 2012 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2166AD159 for ; Wed, 10 Oct 2012 12:19:44 +0000 (UTC) Received: (qmail 76060 invoked by uid 500); 10 Oct 2012 12:19:42 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 75974 invoked by uid 500); 10 Oct 2012 12:19:39 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 75944 invoked by uid 99); 10 Oct 2012 12:19:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 12:19:38 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [91.195.24.3] (HELO mail.open.bg) (91.195.24.3) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 12:19:32 +0000 Received: from [78.83.28.88] (port=49180 helo=eee-az) by mail.open.bg with esmtpsa (Cipher SSL3.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.77) id 1TLvFm-0001hA-0i by authid with login for ; Wed, 10 Oct 2012 15:19:10 +0300 Date: Wed, 10 Oct 2012 15:20:31 +0300 From: svilen To: user@couchdb.apache.org Subject: Re: can i listen to changes of whole server, not per-db? Message-ID: <20121010152031.037dc8f3@eee-az> In-Reply-To: References: <20121007085933.472ff68a@svilendobrev.com> <20121007112532.22743daa@svilendobrev.com> <20121007133712.275a802c@svilendobrev.com> X-Mailer: Claws Mail 3.8.1 (GTK+ 2.24.10; i486-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable ah yes i also had the idea of summarizing all dbs into one just to aggregate the _changes, and abandoned it. For now i'm walking a set of databases and give them some limited attention that handles changes. It very much looks like web-framework routing and request-handling ;-) add some FSMs.. Was even thinking to setup it that way, and to have a dispatcher kick proper urls according to database+message-type-and-state, i.e. translating from stream of changes into stream of commands-as-urls which would be plain http served.. svil On Wed, 10 Oct 2012 10:42:27 +0100 Martin Higham wrote: > We had a similar problem. Initially we had each database replicate > appropriate documents (via a replication filter) to a queue database > and just listened for changes to that. The resource overhead for this > when most databases are dormant didn't make make sense. So we now > have a custom handler that invokes some python whenever a PUT, PUSH > or DELETE occur to a subset of databases. I've put this up on GitHub - > https://github.com/ocastalabs/couchdb_task_handler. >=20 > This handler is dependant on CouchDB including the document ID in the > response headers which it currently doesn't so we had to make a small > change to the CouchDB source. You can find this at > https://github.com/ocastalabs/couchdb/tree/id_and_rev_in_response_headers. > We've raised a Pull Request for this ( > https://github.com/apache/couchdb/pull/32) and it would be great to > get it into 1.3 >=20 > Martin >=20 >=20 >=20 > On 7 October 2012 11:37, svilen wrote: >=20 > > no worries, not that in a hurry. > > i'll put some workaround in the mean time.. > > still prototyping anyway. > > > > ciao > > svil > > > > On Sun, 7 Oct 2012 12:18:29 +0200 > > Benoit Chesneau wrote: > > > > > On Sun, Oct 7, 2012 at 10:25 AM, svilen > > > wrote: > > > >> On 7 October 2012 07:59, svilen wrote: > > > >> > g'day > > > >> > > > > >> > i'm going to have thousands of databases. They are all of > > > >> > similar kind, but belong to different (groups of) users, so > > > >> > can't be bundled into one. And setupping 10000 connections to > > > >> > couchdb/dbXXXXX/_changes doesn't seem very neat, even if on > > > >> > same machine. > > > > > > > > or maybe i can traverse all of them (i think i can figure out > > > > the db names), paying them some limited attention each - e.g. > > > > use short-poll with limit 10 changes.. poor-man's time-share > > > > scheduler ;-) > > > > > > > > svil > > > > > > Well it will be costly, depending on the number of databases you > > > have. If you need it fast you can use rcouch waiting that I port > > > the code to couch. Can be done next week. > > > > > > - beno=C3=AEt > >