From couchdb-user-return-1522-apmail-incubator-couchdb-user-archive=incubator.apache.org@incubator.apache.org Tue Oct 14 01:06:09 2008 Return-Path: Delivered-To: apmail-incubator-couchdb-user-archive@locus.apache.org Received: (qmail 15783 invoked from network); 14 Oct 2008 01:06:08 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 14 Oct 2008 01:06:08 -0000 Received: (qmail 82341 invoked by uid 500); 14 Oct 2008 01:06:09 -0000 Delivered-To: apmail-incubator-couchdb-user-archive@incubator.apache.org Received: (qmail 81922 invoked by uid 500); 14 Oct 2008 01:06:08 -0000 Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: couchdb-user@incubator.apache.org Delivered-To: mailing list couchdb-user@incubator.apache.org Received: (qmail 81909 invoked by uid 99); 14 Oct 2008 01:06:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 13 Oct 2008 18:06:08 -0700 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of ara.t.howard@gmail.com designates 66.249.82.237 as permitted sender) Received: from [66.249.82.237] (HELO wx-out-0506.google.com) (66.249.82.237) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 14 Oct 2008 01:04:59 +0000 Received: by wx-out-0506.google.com with SMTP id s13so825566wxc.21 for ; Mon, 13 Oct 2008 18:05:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=tk7+ZJWDuzVuiXf+zh9vQSI6C87A+fV92RDMiZXTVzE=; b=qKR0AgJ/p2QBqK1HsTpMX4tj05vkYJGll1u63Mx/NSjxuOdVHjbc5XqT/eThDLjqpg kNy5NQaUU07uVChVQ/E22n5AFgoJ+DixRV3miSN1HEDujU5K1BHOb+mdmLPOCYTHkfVE UHRwI0MEZiPnh9PWQeKeEQZ0nSM6W4zMfiejw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=vVs32FuH937YxnQ7H56vEbdnjjL3FZ3F+UzLfwDU3PgWqE7aEeLoBEBdX2VJrBTYoX BEkBVCb99IFmwtF01l7Sm+BBWms7InDEUKERTWVn+aQ8ZfsTMlYdKoPncMnTNXCFfWoA edfnDFec+wggjw6lC5z7vtlvwKSkH57ik5jDI= Received: by 10.70.39.10 with SMTP id m10mr7258609wxm.2.1223946318948; Mon, 13 Oct 2008 18:05:18 -0700 (PDT) Received: from ?192.168.0.3? ([65.103.96.46]) by mx.google.com with ESMTPS id h16sm8014459wxd.21.2008.10.13.18.05.16 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 13 Oct 2008 18:05:17 -0700 (PDT) Message-Id: <25A49021-0F4A-4B39-9C4A-FEFDC0B3738F@gmail.com> From: "ara.t.howard" To: couchdb-user@incubator.apache.org In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Subject: Re: When to use CouchDB, when not to... Date: Mon, 13 Oct 2008 19:04:59 -0600 References: <521183240810111938s2d60320fo30e47cdf22acc7d4@mail.gmail.com> <521183240810121228u3eef309dw7e44141701401e4e@mail.gmail.com> <7E19FE01-58AF-40A6-A785-4A6E9F3E4D51@gmail.com> <27d8d0930810131608r52b5b1bobcd6f75259a867f0@mail.gmail.com> <10F0F911-0B13-4D4A-9566-62BA3E420F62@gmail.com> X-Mailer: Apple Mail (2.929.2) X-Virus-Checked: Checked by ClamAV on apache.org On Oct 13, 2008, at 5:50 PM, Mike Malone wrote: >> i'd contend, however that you'd really have to structure you app in >> a very >> bad way to have any the db schema be of consequence. there are >> indexes and >> there is caching at many levels. hitting the app and db for ever >> request >> doesn't scale period. in fact, rdbms don't scale well anyhow - not >> if you >> count 'easily' in your criteria that is. anyone who has every >> tried tried >> to setup high-availability db backends knows this. again though, >> this is a >> point where couchdb could really shine - they look to have >> addressed this >> from the get go. >> > > I'm not sure I follow... seems like you're contradicting yourself > here a > bit. I agree with the second part - RDBMSs don't scale well. At a high > level, for most web applications they're more or less a single > "thread," > creating a bottleneck in a system that can otherwise scale > horizontally. Of > course you can shard, creating additional "threads," but that's not > a at all > a straightforward process. > > With that in mind, I'd say that you have to structure your > application very > carefully in order for the DB layer to scale. The DB schema is > critical > here, you must denormalize and carefully shard your data in order to > scale. > Normalizing your schema to the extent you described earlier, in > order to > allow "dynamic properties," would destroy performance on a highly > trafficed > web application. Sure, you can cache, but you still need to read > from the DB > for new or invalidated items. > > The sort of architecture I've been contemplating is one where certain > heavyweight entities in the application would be stored in CouchDB > while > other lighter weight/more relational entities would remain in the > RDBMS. > Suppose I were designing a messaging system, for instance. I may > store basic > User information in an RDBMS, along with social graph information, > and an > "inbox" table that stores the id of each Message sent to each User > (some of > these tables would still need to be sharded at some point). The > Messages > themselves I'd store in CouchDB. I may also put certain extended > Profile > information for Users in CouchDB. > > Anyways, I'm fairly new to document-based databases and my intuition > may be > wrong here (please correct me if it is) but it seems to me that > CouchDB can > be used alongside a more traditional RDBMS in this way. > > Mike sure - i agreed totally. by 'does not scale' i meant once you want to move beyond one machine - then it's such a royal PITA. of course the schema is important at a high level, but a relationship here or there shouldn't make or break performance. i'm working on a similar mixture of storage at the moment - but the thought of replication and scaling makes me a bit queasy. the beauty of couchdb is the concept that everything talks http which should provide mechanisms for really novel scaling strategies to emerge. cheers. a @ http://codeforpeople.com/ -- we can deny everything, except that we have the possibility of being better. simply reflect on that. h.h. the 14th dalai lama