couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michel Meyers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1576) Allow deletions to expire after set time
Date Thu, 25 Oct 2012 10:13:12 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13484016#comment-13484016
] 

Michel Meyers commented on COUCHDB-1576:
----------------------------------------

Thanks for the comments. The _revs_limit appears to be a per-document setting (set on the
DB) so it doesn't help in getting rid of old deleted documents (the documents themselves having
been deleted, they never exceed the _revs_limit and remain in the DB forever).

As for rotating the DBs: That's possible but requires an outage across all replicas (of the
entire service) to prevent new deletions from coming in and not being applied to the clean
replica. If there was a way to do a filtered replication of only the documents and NEW deletions,
this could work live, but I'm not sure that's possible.

It's not a big issue, but the perpetual growth of the database to keep these deletions does
make it more difficult when you try to create a new replica in a remote location. A lot of
data is being checked and transmitted for deleted stuff that no one cares about any more and
could just be pruned instead. (In my case there's about a million deletion records amounting
to 500MB of data...)

Compaction would be the way to recover space, except it too keeps the "deletion stubs". (That's
my primary issue: The deletions never ever go away on their own.)
                
> Allow deletions to expire after set time
> ----------------------------------------
>
>                 Key: COUCHDB-1576
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1576
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>    Affects Versions: 1.2
>            Reporter: Michel Meyers
>
> Currently, CouchDB keeps all deletions forever. On a very active database this can lead
to the number of deletions actually exceeding the number of active documents.
> Lotus Domino solves this by having a cut-off date after which deletion stubs are purged
from a replica. It would be great to have a similar functionality in CouchDB, e.g. have a
setting that determines how long deletion stubs are kept and then purges them from the local
copy. Whether this setting resides in the DB design, and thus replicates everywhere, or possibly
an INI setting for the local CouchDB instance is still to be determined.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message