couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Friesen <dan_the_...@telus.net>
Subject Re: Trying to see if CouchDB is right for our project...
Date Tue, 17 Mar 2009 23:07:32 GMT
Dean Landolt wrote:
> On Mon, Mar 16, 2009 at 9:10 PM, Daniel Friesen <dan_the_man@telus.net>wrote:
>
> I can tell you it'd be a whole lot easier to just return every object for an
> entire page (if you store the root page with each document). It fits much
> nicer with the model of views -- and you can always filter on the client
> side. What kind of scale are we talking about though? Could there be
> thousands of widgets on a page? If so, some smaller logical grouping could
> be considered and you could just pull in what you need with a multi-key
> request.
>   
Widget is actually just a general word I threw out. We actually call 
them "jits" in our app because quite simply they do everything. A "jit" 
ranges from anything to a container for stuff, to a blob of text, and 
image tag, a button, a calendar, an accordion, etc... And a very large 
number of them appear on a page (go to a random webpage (I suppose not 
tooo complex), count up the number of divs or whatever elements on the 
page, and then you'll get a rough number of how many jits will be on a 
page).
I haven't bothered to benchmark how many SQL calls we're actually making 
on every single pageview currently, but quite simply I already know that 
we're making an extreme amount of them just because we have to query for 
data so we can query for data dependent on it.

Bradford Winfrey wrote:
> Just thinking aloud here, but storing each widget (if as Dean asked there are in fact
"lots" of them) would be much better suited to be stored in its own document, should a widget
change, you'd save a significant amount of space (since couch will keep a full copy of each
revision until compaction) rather than saving the whole collection of them each time.  
>
> Just a thought...
> Brad
>   
I intend to, that was just a structure example. I've looked at CouchDB a 
bit more and decided the best idea was to store widgets in their own 
documents. Not just because of space, but for writing (like the comments 
example on the couchdb-joins article), and because jits don't really fit 
in the same document.


Chris Anderson wrote:
>> ...
>>     
>
> Taking this approach would probably require that you store the full
> path(s) to the widget on the widget document itself. This isn't so bad
> but does complicate changing only a parent node say, halfway up the
> tree, as you have to touch all it's subnodes.
>   
Not really possible. There is no full path to the widget in the 
document. Quite simply because widgets don't belong to a document (I 
have limited it so that widgets can't cross sites) but a widget can 
belong to multiple parent widgets and as a result can belong to multiple 
pages.
The ability to put the same Widgets on multiple pages is essential to 
the app we're building so it can't really be worked around.

I actually drastically simplified the example. We actually have another 
level, "references" which make it possible to have multiple instances of 
the same jit instance. So something could actually end up like:

page:A hasJit ref:AAA refersto jit:BBB hasJit ref:CCC refersto jit:DDD
                               ^^^^^^^ hasJit ref:EEE refersto jit:FFF

page:B hasJit ref:GGG refersto jit:HHH hasJit ref:III refersto jit:FFF

                                                               jit:FFF ...some jit data...

A jit reference normally stores only things like what jit it refers to, 
and other things like sortkeys which don't belong to the jit, but rather 
belong to it's individual existence on a page. While the Jit itself 
stores things like what type of jit it is (container, text, calender, 
etc...), what jits are inside of it (if the jit type can support stuff 
inside it; container, accordion, etc...), jit level access control (a 
jit is editable if you can edit the site, you can edit the page, or you 
can edit the jit itself), and a jit "state" which is basically a 
key/value hash of data which the jit type is responsible for managing 
and turning into the actual output.


The only possibilities I can think of, are a view with the ability to 
use data from other documents it knows the id for. Or a plugin that 
makes it possible to write a "walk" function similar to the code in a 
view (ie: JS).
So what it looks like, is I need a database engine with a good 
lightweight scripting capability builtin for mangling data into query 
output to avoid doing all that work in the app itself and killing both 
with the insane amount of extra traffic.

~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://nadir-seen-fire.com]
-Nadir-Point & Wiki-Tools (http://nadir-point.com) (http://wiki-tools.com)
-MonkeyScript (http://monkeyscript.org)
-Animepedia (http://anime.wikia.com)
-Narutopedia (http://naruto.wikia.com)
-Soul Eater Wiki (http://souleater.wikia.com)


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message