incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Compaction of a database with the same number of documents is getting slower over time
Date Tue, 22 Apr 2014 14:15:07 GMT
Sorry, just so we're super clear -- the "doc_count" is roughly constant but the "doc_del_count"
is rising? Compaction time scales with the sum of those two values. Your options to get it
back down under control at the moment are

1) Purge the deleted docs (lots of caveats about replication, potential for view index resets,
etc.)
2) Rotate over to a new database, either by querying both or by replicating non-deleted docs
to the new one

Neither one is particularly palatable. CouchDB currently keeps the tombstones around forever
so that replication can always work. Making changes on that front is a pretty subtle thing
but maybe not completely impossible.

Also, there's a new compactor in the works that is faster and generates smaller files.

Adam

On Apr 22, 2014, at 9:46 AM, Boaz Citrin <bcitrin@gmail.com> wrote:

> Yes, the documents change quickly, Many deletions and insertions, but total
> number don't change that much.
> 
> 
> On Tue, Apr 22, 2014 at 4:41 PM, Adam Kocoloski <adam.kocoloski@gmail.com>wrote:
> 
>> To first order it shouldn't be the case.
>> 
>> Is the number of deleted documents growing with time? That'd have an
>> impact.
>> 
>> Adam
>> 
>>> On Apr 22, 2014, at 8:06 AM, Boaz Citrin <bcitrin@gmail.com> wrote:
>>> 
>>> Hello,
>>> 
>>> Our database contains more or less the same number of documents, however
>>> the documents themselves change frequently.
>>> I would expect that compaction time will be the same, but I see that over
>>> time it takes longer to compact the database.
>>> 
>>> Why is it so? Any way to overcome this?
>>> 
>>> Thanks,
>>> 
>>> Boaz
>> 


Mime
View raw message