couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Smith <jason.h.sm...@gmail.com>
Subject Re: Purging documents and view invalidation
Date Fri, 05 Jul 2013 09:21:06 GMT
I slightly disagree with Bob, but he is right that all purge buys you (vs.
filtered replication and then swapping DBs) is a little bit of uptime.
Purge is not "untested" but it is rarely used in the wild, so the
cost/benefit for your uptime is something between "risky" and "unknown."

(For me, personally, I would purge.)


On Fri, Jul 5, 2013 at 3:31 PM, Robert Newson <rnewson@apache.org> wrote:

> Paul,
>
> If you replicate this database to another database and use a filter
> that blocks deleted documents, the target will not contain a trace of
> your 100 million deletes (that is, you can build a new database
> without cruft without messing with your existing database). During the
> replication, you can query the view on the target to build it
> incrementally, or wait till the end, query it once and wait for
> completion. At the end, flip your app to look at the new database
> instead.
>
> The _purge feature is really only for the case where you accidentally
> write your root password down in a document id or something (since
> compaction will sweep away old document contents). I advise against
> using it for any other reason.
>
> B.
>
>
>
>
> On 5 July 2013 09:17, Jason Smith <jhs@apache.org> wrote:
> > Hi, Paul. I wrote up some thoughts on purging here:
> > https://github.com/iriscouch/cqs#purging-couchdb
> >
> > Note, that procedure is untested. It works as a thought experiment only.
> >
> > The procedure looks complicated, but all you will need is the core purge,
> > view, purge, view, etc. cadence as described in Damien's email I linked
> to.
> > As long as you never purge twice before hitting the view, you are fine.
> > Again, to my knowledge, the purge code is less well tested than other
> parts
> > of CouchDB, so perhaps copy your .couch file and try with that until you
> > are confident.
> >
> >
> > On Fri, Jul 5, 2013 at 2:37 PM, Paul Hirst <paul.hirst@sophos.com>
> wrote:
> >
> >> I would like to purge a few (~100 million) documents from my database.
> >> I've been going through deleting them all, and that'll be complete in
> the
> >> next few days but I would like to free up some extra space by purging
> them
> >> also.
> >>
> >> My concern is around a comment on the wiki page here
> >> http://wiki.apache.org/couchdb/Purge_Documents
> >>
> >> 'If you have purged more than one document between querying your views,
> >> you will find that they will rebuild from scratch.'
> >>
> >> Since I have already deleted the documents I know they aren't showing up
> >> in the view any longer. Is there any way I can avoid this view
> >> invalidation? (My views take about 10 days to build from scratch so I
> can't
> >> afford the hit).
> >>
> >> I have a replica of the database. I could do the purge on the replica,
> >> wait for the view to rebuild, switch over, purge on the original db,
> wait
> >> for the view, switch back, unless there are any obvious problems with
> this
> >> approach?
> >>
> >> Cheers,
> >> Paul
> >>
> >> ________________________________
> >>
> >> Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP,
> >> United Kingdom.
> >> Company Reg No 2096520. VAT Reg No GB 991 2418 08.
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message