couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sho Fukamachi <>
Subject Re: CouchDB/Memcached
Date Mon, 21 Jul 2008 20:08:27 GMT

On 22/07/2008, at 3:45 AM, Bradford Winfrey wrote:

> Half of me is really eager to see this, the other half is worrying  
> that it's going to be great and is going to make me want to re- 
> factor my application away from DM.

Ha, you and me both, and that after about 2 weeks of non stop  
communication with the (fantastic) DM crew getting dm-couch working  
properly on 0.8.0...

Many things about DM are really nicely done - its .auto_migrate!  
function alone has saved me probably *hours*. And it's nice to have  
something that "just works", albeit slowly, when you're converting a  
pre-existing app. Just getting anything at all up and running is a  
huge morale booster.

But the limitations exist, to be sure. One which has really bugged me  
is the requirement to define a full "schema" in the model, along with  
data types, and the inability to save arbitrary (ie, undefined)  
fields... these things seem like detritus left over from RDBMS schema  
necessities and just bug me. I've already started mixing in my own  
pure rest-client functions just so I can do things like step outside  
the DM "hard schema". I'm not writing my own yet since I don't even  
know what I want it to do but the writing is becoming visible on the  
wall : )

I'm also very keen to see alternatives, if for no reason other than to  
see other people's ideas along the same lines.

Gotta admit though that Couch's REST model is so light that pretty  
much anything seems heavy on top of it ..

> Nicely done.
> ----- Original Message ----
> From: Troy Kruthoff <>
> To:
> Sent: Monday, July 21, 2008 12:24:39 PM
> Subject: Re: CouchDB/Memcached
> dm is a very cool project, and I've used it for RDBMS stuff, but for
> couch we ended up rolling our own because I felt the dm adapter API is
> too confining to exploit the potential of couch.  So we  are working
> on the database layer I mentioned below and a mapper layer as well.
> The mapper layer is very similar to dm, but we are trying to take full
> advantage of not being tied to a schema.  Our intent is to open source
> the projects in the next few weeks.
> -- troy
> On Jul 21, 2008, at 9:51 AM, Bradford Winfrey wrote:
>> Troy,
>> My implementation is in Ruby as well and my greatest concern is
>> using large memory hashes, your approach makes sense to abstract it
>> in such a way.  Currently I'm using DataMapper as my intermediary
>> and have thought many, many times about scrapping it and rolling
>> something internally so I don't have all of the added overhead (of
>> which, there isn't much, but there is some, it's still very fast
>> IMHO) when having to deal with merging the multi-key needs together.
>> I can't wait for the docs to get published, I'm very intrigued.
>> ----- Original Message ----
>> From: Troy Kruthoff <>
>> To:
>> Sent: Monday, July 21, 2008 11:36:14 AM
>> Subject: Re: CouchDB/Memcached
>> Bradford,
>> We are still hacking through this, but I plan to post our code either
>> on a blog or the couchdb wiki once we get it smoothed out a little.
>> What we have done is abstracted the REST API in our application via  
>> an
>> object we call "database".  So in our code (we use Ruby) we use:
>> database.get('id') or database.get(['id1','id2','id3',...])
>> where database.get:
>> 1) looks up each id in an in-memory hash table
>> 2) queries memcache for all id's not found in hash (memcache can use
>> multi-get here) (adds results to hash)
>> 3) queries couch for all id's not found in memcache (adds results to
>> memcache and hash)
>> and of course put,post and delete are abstracted as well to update/
>> delete the hash and memcached.
>> We also use a memory session concept for all updates/writes.  So when
>> we create or edit a document and save it, we store the document in  
>> the
>> hash (generating the in the application for new docs) and mark
>> it as dirty.  Then the app calls database.commit which uses the couch
>> bulk_docs api to commit the changes to couch.
>> We have discussed internally, implementing the above in C as an open
>> source "couch driver" with Ruby, Python and PHP language bindings.
>> -- troy
>> On Jul 21, 2008, at 6:08 AM, Bradford Winfrey wrote:
>>> Now that it's been mentioned a few times - does anyone have
>>> any hints/tips for working out memcached with couchdb?  I think a
>>> gentleman named Troy said that's how he was getting around this
>>> problem
>>> in emails prior to this discussion - maybe we should branch this  
>>> to a
>>> new thread?
>>> *I sense a wiki page being authored*

View raw message