From couchdb-user-return-875-apmail-incubator-couchdb-user-archive=incubator.apache.org@incubator.apache.org Mon Jul 21 17:25:15 2008 Return-Path: Delivered-To: apmail-incubator-couchdb-user-archive@locus.apache.org Received: (qmail 66853 invoked from network); 21 Jul 2008 17:25:15 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 21 Jul 2008 17:25:15 -0000 Received: (qmail 50274 invoked by uid 500); 21 Jul 2008 17:25:14 -0000 Delivered-To: apmail-incubator-couchdb-user-archive@incubator.apache.org Received: (qmail 50245 invoked by uid 500); 21 Jul 2008 17:25:13 -0000 Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: couchdb-user@incubator.apache.org Delivered-To: mailing list couchdb-user@incubator.apache.org Received: (qmail 50234 invoked by uid 99); 21 Jul 2008 17:25:13 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Jul 2008 10:25:13 -0700 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.198.242] (HELO rv-out-0708.google.com) (209.85.198.242) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 21 Jul 2008 17:24:19 +0000 Received: by rv-out-0708.google.com with SMTP id k29so1177989rvb.0 for ; Mon, 21 Jul 2008 10:24:43 -0700 (PDT) Received: by 10.141.83.15 with SMTP id k15mr1971064rvl.289.1216661083693; Mon, 21 Jul 2008 10:24:43 -0700 (PDT) Received: from ?192.168.1.13? ( [64.113.107.88]) by mx.google.com with ESMTPS id c20sm7146175rvf.1.2008.07.21.10.24.41 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 21 Jul 2008 10:24:42 -0700 (PDT) Message-Id: <90BD2C01-9F62-4277-9B75-5ED94232E802@blit.com> To: couchdb-user@incubator.apache.org In-Reply-To: <488863.99532.qm@web57207.mail.re3.yahoo.com> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v919.2) Subject: Re: CouchDB/Memcached Date: Mon, 21 Jul 2008 10:24:39 -0700 References: <488863.99532.qm@web57207.mail.re3.yahoo.com> X-Mailer: Apple Mail (2.919.2) From: Troy Kruthoff X-Virus-Checked: Checked by ClamAV on apache.org dm is a very cool project, and I've used it for RDBMS stuff, but for couch we ended up rolling our own because I felt the dm adapter API is too confining to exploit the potential of couch. So we are working on the database layer I mentioned below and a mapper layer as well. The mapper layer is very similar to dm, but we are trying to take full advantage of not being tied to a schema. Our intent is to open source the projects in the next few weeks. -- troy On Jul 21, 2008, at 9:51 AM, Bradford Winfrey wrote: > Troy, > > My implementation is in Ruby as well and my greatest concern is > using large memory hashes, your approach makes sense to abstract it > in such a way. Currently I'm using DataMapper as my intermediary > and have thought many, many times about scrapping it and rolling > something internally so I don't have all of the added overhead (of > which, there isn't much, but there is some, it's still very fast > IMHO) when having to deal with merging the multi-key needs together. > > I can't wait for the docs to get published, I'm very intrigued. > > > > ----- Original Message ---- > From: Troy Kruthoff > To: couchdb-user@incubator.apache.org > Sent: Monday, July 21, 2008 11:36:14 AM > Subject: Re: CouchDB/Memcached > > Bradford, > > We are still hacking through this, but I plan to post our code either > on a blog or the couchdb wiki once we get it smoothed out a little. > What we have done is abstracted the REST API in our application via an > object we call "database". So in our code (we use Ruby) we use: > > database.get('id') or database.get(['id1','id2','id3',...]) > > where database.get: > > 1) looks up each id in an in-memory hash table > 2) queries memcache for all id's not found in hash (memcache can use > multi-get here) (adds results to hash) > 3) queries couch for all id's not found in memcache (adds results to > memcache and hash) > > and of course put,post and delete are abstracted as well to update/ > delete the hash and memcached. > > We also use a memory session concept for all updates/writes. So when > we create or edit a document and save it, we store the document in the > hash (generating the doc.id in the application for new docs) and mark > it as dirty. Then the app calls database.commit which uses the couch > bulk_docs api to commit the changes to couch. > > We have discussed internally, implementing the above in C as an open > source "couch driver" with Ruby, Python and PHP language bindings. > > -- troy > > > > > > > > > > > On Jul 21, 2008, at 6:08 AM, Bradford Winfrey wrote: > >> Now that it's been mentioned a few times - does anyone have >> any hints/tips for working out memcached with couchdb? I think a >> gentleman named Troy said that's how he was getting around this >> problem >> in emails prior to this discussion - maybe we should branch this to a >> new thread? >> >> *I sense a wiki page being authored* >> >> > >