couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <>
Subject Re: Trying to see if CouchDB is right for our project...
Date Tue, 17 Mar 2009 23:14:52 GMT
On Tue, Mar 17, 2009 at 4:07 PM, Daniel Friesen <> wrote:
> Dean Landolt wrote:
>> On Mon, Mar 16, 2009 at 9:10 PM, Daniel Friesen
>> <>wrote:
>> I can tell you it'd be a whole lot easier to just return every object for
>> an
>> entire page (if you store the root page with each document). It fits much
>> nicer with the model of views -- and you can always filter on the client
>> side. What kind of scale are we talking about though? Could there be
>> thousands of widgets on a page? If so, some smaller logical grouping could
>> be considered and you could just pull in what you need with a multi-key
>> request.
> Widget is actually just a general word I threw out. We actually call them
> "jits" in our app because quite simply they do everything. A "jit" ranges
> from anything to a container for stuff, to a blob of text, and image tag, a
> button, a calendar, an accordion, etc... And a very large number of them
> appear on a page (go to a random webpage (I suppose not tooo complex), count
> up the number of divs or whatever elements on the page, and then you'll get
> a rough number of how many jits will be on a page).
> I haven't bothered to benchmark how many SQL calls we're actually making on
> every single pageview currently, but quite simply I already know that we're
> making an extreme amount of them just because we have to query for data so
> we can query for data dependent on it.
> Bradford Winfrey wrote:
>> Just thinking aloud here, but storing each widget (if as Dean asked there
>> are in fact "lots" of them) would be much better suited to be stored in its
>> own document, should a widget change, you'd save a significant amount of
>> space (since couch will keep a full copy of each revision until compaction)
>> rather than saving the whole collection of them each time.
>> Just a thought...
>> Brad
> I intend to, that was just a structure example. I've looked at CouchDB a bit
> more and decided the best idea was to store widgets in their own documents.
> Not just because of space, but for writing (like the comments example on the
> couchdb-joins article), and because jits don't really fit in the same
> document.
> Chris Anderson wrote:
>>> ...
>> Taking this approach would probably require that you store the full
>> path(s) to the widget on the widget document itself. This isn't so bad
>> but does complicate changing only a parent node say, halfway up the
>> tree, as you have to touch all it's subnodes.
> Not really possible. There is no full path to the widget in the document.
> Quite simply because widgets don't belong to a document (I have limited it
> so that widgets can't cross sites) but a widget can belong to multiple
> parent widgets and as a result can belong to multiple pages.
> The ability to put the same Widgets on multiple pages is essential to the
> app we're building so it can't really be worked around.

This is possible, as long as you are willing to eat up a fair amount
of disk space caching *all possible* paths to the jit. I guess I'm
essentially talking about running your "walk" function and then
memoizing the results of it [an array of paths] on the jit document
itself. This should be possible as long as the graph of nested jits is

The queries from such a view would be pretty clean. For a given root
page-id you'd be able to pull in all and only the jits on that page
(sorted according to their nesting order) in a single request.

> I actually drastically simplified the example. We actually have another
> level, "references" which make it possible to have multiple instances of the
> same jit instance. So something could actually end up like:
> page:A hasJit ref:AAA refersto jit:BBB hasJit ref:CCC refersto jit:DDD
>                              ^^^^^^^ hasJit ref:EEE refersto jit:FFF
> page:B hasJit ref:GGG refersto jit:HHH hasJit ref:III refersto jit:FFF
   jit:FFF ...some
> jit data...
> A jit reference normally stores only things like what jit it refers to, and
> other things like sortkeys which don't belong to the jit, but rather belong
> to it's individual existence on a page. While the Jit itself stores things
> like what type of jit it is (container, text, calender, etc...), what jits
> are inside of it (if the jit type can support stuff inside it; container,
> accordion, etc...), jit level access control (a jit is editable if you can
> edit the site, you can edit the page, or you can edit the jit itself), and a
> jit "state" which is basically a key/value hash of data which the jit type
> is responsible for managing and turning into the actual output.
> The only possibilities I can think of, are a view with the ability to use
> data from other documents it knows the id for. Or a plugin that makes it
> possible to write a "walk" function similar to the code in a view (ie: JS).
> So what it looks like, is I need a database engine with a good lightweight
> scripting capability builtin for mangling data into query output to avoid
> doing all that work in the app itself and killing both with the insane
> amount of extra traffic.
> ~Daniel Friesen (Dantman, Nadir-Seen-Fire) []
> -Nadir-Point & Wiki-Tools ( (
> -MonkeyScript (
> -Animepedia (
> -Narutopedia (
> -Soul Eater Wiki (

Chris Anderson

View raw message