couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcus Persson Lindqvist <marcus.pers...@gmail.com>
Subject _mutiple_ databases memory profile
Date Sun, 14 Jun 2009 21:03:21 GMT
Greetings all!

In short - what factors are involved in memory consumtion for couchdb for a
large (x * 1000+) number of databases? Any hints welcome.



I've recently starting to dig couchdb alot and are using it as primary
storage of a backend-type application to much success and relaxation. It
really saves a lot of pain not having to care much about the details of a
repository.

Now, however, my application is growing in data and I'm looking for some
pointers of what to expect in terms of memory consumption (my primary
bottleneck).

The data is highly segmentized - I'm using about 4 different "classes" of
documents from X different "sources" (X is currently 200 but might grow up
to 2000 or more), neither of which need to know about the others. Going
reduction of btrees and such, I figured I would use a separate database for
each, yielding 800 DBs at the moment.

And kudos to couch for making it a breeze implementing, it was really nice
and smooth.

But now I'm starting to see some memory consumtion growth and I'm looking
for pointers of how to think about this. What mechanisms actually cosumes
memory? What should one avoid? Is it better to use fewer databases for this
point of view.

What would be a reasonable memory footprint and how does one caclulate on
it? Currently it consumed about 300MB.

Each database is really just a pet store. I need to extract documents in
order. Thats it. I'm currently doing this with a simple view. (Are there any
"trivial" build-in way of getting documents i reversed insertion-order btw?)

And yeah, the load for most databases is really low, so insert/output
performance could be compromized for less memory consumtions.

Any hints, tips or experiences?

Marcus

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message