couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CGS <cgsmcml...@gmail.com>
Subject Re: Compactation and resource fighing problems
Date Thu, 06 Oct 2011 07:14:12 GMT
Hi,

Unfortunately, there isn't too much to do about but to change a bit the
design of your DB. That means:

Option 1:
Change from continuous writing to bulk writing. That means you should create
a buffer for your writers in your application and when the buffer is full,
flush it to DB by the mean of bulk operation. In this way your writers will
allow DB to do some other operations in between bulk operation requests and
the writing time gets shorter.

Option 2:
Divide et impera. Try to divide your DB in smaller DB's and run
administrative DB operations when there is not so much writing in that DB.

There may be some other options, but I stop to these two options.

Cheers,
CGS



On Wed, Oct 5, 2011 at 9:26 PM, Pau Freixes <pfreixes@gmail.com> wrote:

> Hi to all,
>
> Sorry for the subject, could be funny but it's a way to describe in a
> few words some problems with my compaction problems, I can try to
> explain.
>
> Im running a Couchdb instance with only one data base with a some
> thousands of documents, every document has only a 1k or lesser of
> size. B I have only a one aplication writting and updating 20
> documents/second mantained over time and in serial way. I have also
> some readers, but may be negligible
>
> The data base increase and eat a lot of space, growing from 0.5 Gbytes
> to 2 or more 3 Gb in only few hours. Every hour I run a compaction
> task, and here begin the problems.
> I have been trying to limit the maxim number of revisions to two, but
> it has been impossible.
>
> When compactation task run the write aplication decrease the updating
> documents/second but not disturbing. But the problem is when
> compactation task try to swap old file to new compactation file. When
> it's happen the task compactation run in some wait state and the write
> aplication decrease the updating document/seconds down to the few
> units and the queue grow delaying each write.
>
> Anyone have a solution to this problem?
>
> --
> --pau
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message