From user-return-7046-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Fri Oct 23 22:54:31 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 26438 invoked from network); 23 Oct 2009 22:54:31 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 23 Oct 2009 22:54:31 -0000 Received: (qmail 39959 invoked by uid 500); 23 Oct 2009 22:54:30 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 39902 invoked by uid 500); 23 Oct 2009 22:54:30 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 39892 invoked by uid 99); 23 Oct 2009 22:54:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Oct 2009 22:54:30 +0000 X-ASF-Spam-Status: No, hits=-1.6 required=5.0 tests=BAYES_00,FS_REPLICA X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of truemped@googlemail.com designates 72.14.220.153 as permitted sender) Received: from [72.14.220.153] (HELO fg-out-1718.google.com) (72.14.220.153) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 23 Oct 2009 22:54:27 +0000 Received: by fg-out-1718.google.com with SMTP id 22so401815fge.5 for ; Fri, 23 Oct 2009 15:54:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:received:content-type :mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=G4m5LMYWjqX+n8oOLIjTHZwYOSP3VFubyYIYFDeaXXo=; b=mgTHrhsM5vc7FmPhUXWUX8zpGdAB3uon2oxICqG7KJDo8ACmktheyIRRgz/UZedIJw llz6roMyiojulXIk+TxtCW8ZjttSpe6K5SJ0uSzAFGbdczC2iXEgjwOO5oHwRKLNRYy4 EJ6cypmKpz/R4MF57CFB17dQL7o+gGevytXrs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; b=AQAfkRwRtsp1kgZ6J1UldILX88LUHjB9Uac8Q9/bO4T+GJCv03gSIANtkR4qUEVVNT 8JU7eHGlB700CPmLqw0m+zMSxKT0FgqJy8Pn6XPbmvIHVijgYsohtGqpTanNpQp19J+Y sN4JnSKrgyDWiOsSPmP5kAGCKOuGsoHYeEUkQ= Received: by 10.86.158.5 with SMTP id g5mr6192431fge.32.1256338446032; Fri, 23 Oct 2009 15:54:06 -0700 (PDT) Received: from hence22.org ([85.131.190.91]) by mx.google.com with ESMTPS id l12sm4646857fgb.17.2009.10.23.15.54.04 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 23 Oct 2009 15:54:05 -0700 (PDT) Received: from saturn.duvn.priv (e178121201.adsl.alicedsl.de [85.178.121.201]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by hence22.org (Postfix) with ESMTPSA id 900763E479 for ; Sat, 24 Oct 2009 00:43:36 +0200 (CEST) Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes Mime-Version: 1.0 (Apple Message framework v1076) Subject: Re: massive replication? From: =?iso-8859-1?Q?Daniel_Tr=FCmper?= In-Reply-To: <4AE2247B.1010803@meetinghouse.net> Date: Sat, 24 Oct 2009 00:43:45 +0200 Content-Transfer-Encoding: 7bit Message-Id: References: <4AE2028F.9030502@meetinghouse.net> <0C6A665F-53C0-41ED-9A8E-C00098B45174@apache.org> <4AE20CD6.7010500@meetinghouse.net> <4AE2247B.1010803@meetinghouse.net> To: user@couchdb.apache.org X-Mailer: Apple Mail (2.1076) >>> 2. It seems like there's a point at which explicit 1-1 replication >>> starts to be an administrative nightmare. Some kind of publish- >>> subscribe or multi-cast update model seems needed. >> Would the new continuous replication feature be what you need? With >> this all changes to A get automatically replicated to B, if I get >> things right here... > Well it's more of, when I change A, I want the changes to propagate > to B through Z (or "all") - with some sort of multi-cast addressing > rather than having to identify every node explicitly. Hm, I guess you would have to create/write/program a custom layer on top of each CouchDB instance. Maybe an adpted version of the couchdb- proxy [1] that would listen on the multicast address. With this you could notice new instances being up and the replication from the master machines would be handled by the proxy, i.e. the proxy would send the replication commands to the master. You could then even do this in layers of proxies and CouchDBs... But: while writing the above lines I notice that you may be want to have a look at Zookeeper [2] !? > ZooKeeper is a centralized service for maintaining configuration > information, naming, providing distributed synchronization, and > providing group services. You could basically wrap the start script and write additional information (like the CouchDB URL) into Zookeeper. One node could then periodically read the info about existing CouchDB instances and trigger or configure the replications... Daniel [1]: http://github.com/benoitc/couchdbproxy [2]: http://hadoop.apache.org/zookeeper/