couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: Pruning a couchdb database
Date Sun, 03 Oct 2010 23:36:17 GMT
It's not that it's 'a bad idea'. The problem is that there is no general
function which maps view keys to document ids. CouchDB would have no idea
which document(s) to DELETE. Even more troublesome is when a single document
emits multiple keys and you ask to delete only some of them. It is entirely
unclear to me how to handle view deletions.
On Oct 3, 2010 2:05 AM, "John Logsdon" <john.logsdon@netdev.co.uk> wrote:
> I have to agree, I'm not happy with managing multiple databases and
querying
> across all of them and aggregating in the client.
>
> I'll try the get view followed by bulk delete ou,t but it seems a
cumbersome
> way to manage sets.
> I rather like using the power of a RESTful Web Service call and doing a
> DELETE on a viewinstead. It seems like a lot less work all round if the
> server doesn't have to stream out the matches and then process one or more
> bulk delete operations.
>
> Is having DELETE on a view such a bad idea?
>
> Regards
>
> John
>
> On 2 October 2010 17:29, Michael Genereux <mgenereu@gmail.com> wrote:
>
>> The problem I have had with rotating databases is that statistics are
>> harder to gather. Let's say one database has an average of $50 profit
>> per widget sold and the other database has an average of $30 profit
>> per widget sold, you can't just add 50 and 30 and divide by two to get
>> the profit over two months. The reason I use CouchDB is to make
>> getting realtime statistics easier.
>>
>> On Mon, Sep 27, 2010 at 5:38 PM, Chris Anderson <jchris@apache.org>
wrote:
>> > On Mon, Sep 27, 2010 at 8:36 AM, Simon Metson
>> > <simonmetson@googlemail.com> wrote:
>> >> Hi John,
>> >> How about having the date in the database name e.g:
>> >>
>> >> johns_database/Sept2010
>> >> johns_database/Oct2010
>> >>
>> >> Pruning then becomes a case of dropping the database en masse, e.g. to
>> kill
>> >> all records in a certain time window.
>> >>
>> >> That said, pulling out a view and doing a bulk delete shouldn't be too
>> slow.
>> >> Also, don't forget to compact the database after you delete records
from
>> it!
>> >
>> > For your use case, rotating databases is much better. Deletes still
>> > linger in the database file (so they can replicate) so the case of
>> > deleting old records won't reclaim 100% of the space. There is a purge
>> > command for truly removing records, but I think the rotating db files
>> > is much simpler.
>> >
>> > Chris
>> >
>> >> Cheers
>> >> Simon
>> >>
>> >> On 27 Sep 2010, at 16:26, John Logsdon wrote:
>> >>
>> >>> Hi
>> >>>
>> >>> I need to hold, potentially, millions of call records in a couchdb
>> cluster
>> >>> and using a management process periodically truncate the records
based
>> on
>> >>> their age. I can see how I can get the Set of matching records using
a
>> >>> View
>> >>> but how do I delete them in the same operation? My understanding is
>> that I
>> >>> can only GET from a View so I cant see how to do this without
bringing
>> the
>> >>> data back to the client and issuing a DELETE for each record which
>> would
>> >>> be
>> >>> a real problem wrt performance.
>> >>>
>> >>> Regards
>> >>>
>> >>> John
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Chris Anderson
>> > http://jchrisa.net
>> > http://couch.io
>> >
>>
>
>
>
> --
> John Logsdon
> CEO
> NetDev Limited
> +44 1273 773661
> +44 7899 811245
> http://www.netdev.co.uk
> Registered in England and Wales
> Company Number 04741258

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message