couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Anderson <jch...@apache.org>
Subject Re: _mutiple_ databases memory profile
Date Sun, 14 Jun 2009 21:11:49 GMT
On Sun, Jun 14, 2009 at 2:03 PM, Marcus Persson
Lindqvist<marcus.persson@gmail.com> wrote:
> Greetings all!
>
> In short - what factors are involved in memory consumtion for couchdb for a
> large (x * 1000+) number of databases? Any hints welcome.
>
>
>
> I've recently starting to dig couchdb alot and are using it as primary
> storage of a backend-type application to much success and relaxation. It
> really saves a lot of pain not having to care much about the details of a
> repository.
>
> Now, however, my application is growing in data and I'm looking for some
> pointers of what to expect in terms of memory consumption (my primary
> bottleneck).
>
> The data is highly segmentized - I'm using about 4 different "classes" of
> documents from X different "sources" (X is currently 200 but might grow up
> to 2000 or more), neither of which need to know about the others. Going
> reduction of btrees and such, I figured I would use a separate database for
> each, yielding 800 DBs at the moment.
>
> And kudos to couch for making it a breeze implementing, it was really nice
> and smooth.
>
> But now I'm starting to see some memory consumtion growth and I'm looking
> for pointers of how to think about this. What mechanisms actually cosumes
> memory? What should one avoid? Is it better to use fewer databases for this
> point of view.
>
> What would be a reasonable memory footprint and how does one caclulate on
> it? Currently it consumed about 300MB.
>
> Each database is really just a pet store. I need to extract documents in
> order. Thats it. I'm currently doing this with a simple view. (Are there any
> "trivial" build-in way of getting documents i reversed insertion-order btw?)
>

If you really just want to pull them out in the order you put them in,
have a look at the _all_docs_by_seq view:

http://127.0.0.1:5984/test_suite_db/_all_docs_by_seq?descending=true

Do note that if you update a document after inserting it, it will move
to a new position in the list.

> And yeah, the load for most databases is really low, so insert/output
> performance could be compromized for less memory consumtions.
>
> Any hints, tips or experiences?
>
> Marcus
>



-- 
Chris Anderson
http://jchrisa.net
http://couch.io

Mime
View raw message