couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Bleigh <mich...@intridea.com>
Subject CouchDB Twitter Clone Architecture
Date Sun, 08 Nov 2009 00:09:32 GMT
So I've been thinking through the architecture of a Twitter-esque
system in Couch as a kind of thought exercise to get a better handle
on some of the more difficult corners of view generation. What would
be the most effective manner of creating Twitter-like status streams?

My initial feeling is to store the followings of a given user as an
array in the user's document and also have a view that compiles the
followers of a given user. When a user posts a status update, the
application would fetch the follower list from that view and simply
attach it to the status document. It is then a simply matter of a
composite key map of a given status document to all of the users
stored within to create a given user's home timeline.

Where this breaks down is your @aplusk scenario. Storing a 3.5 million
entry array with a document is obviously going to cripple performance
(at least I would think it would) as well as take up massive disk
space (I estimated around 7MB for a single JSON status with 1MM
followers).

So if this solution isn't scalable to millions of users, what's an
architecture that would be? How do you compose the user's tweet stream
such that it can be pulled in an efficient manner?

Just trying to start a discussion to help me better understand
document-oriented architecture, feel free to ignore me!

Michael Bleigh

Mime
View raw message