couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stephen bartell <snbart...@gmail.com>
Subject Re: auto delete docs
Date Thu, 30 Aug 2012 17:26:10 GMT
What Simon said.  Im doing this in production.  I have a python script set up as a cron job
which queries the expire_time view.  Docs that get returned get deleted. Its a real small,
simple script and works like a charm.

Im pretty sure you know this, but make sure to page through your view results.  If your database
hasn't been 'cleaned out' for a while and the database is huge, you're going to get quite
a massive response.  I once made the mistake of not paging my results in a node program. 
V8's mem limit is 1gb on a 64 bit machine, so once this limit was exceeded, the program would
crash and never complete its job.  You could see where this leads if that program is responsible
for getting rid of old docs and thus keeping database size under control.

Stephen Bartell

Look ahead, understand the shift, and imagine the right solution five years from now. Then
do it!
-baekdal

On Aug 30, 2012, at 9:58 AM, Stephan Bardubitzki wrote:

> Okay, the link works now. Must have been an issue at Apache.
> 
> On 12-08-30 09:30 AM, Stephan Bardubitzki wrote:
>> @Simon:
>> 
>> Thanks, again. Could you please double check the link, I can't open it.
>> 
>> @Matthieu
>> 
>> Unfortunately, I'm not familiar with Erlang. I need to do this task on node.js with
nano (https://github.com/dscape/nano).
>> 
>> 
>> On 12-08-30 09:03 AM, Simon Metson wrote:
>>> Sure, you can use _bulk_docs and send a json doc like:
>>> 
>>> {
>>>   "docs": [
>>>     {"_id": "expired_doc_0", "_rev": "1-62657917", "_deleted": true},
>>>     {"_id": "expired_doc_1", "_rev": "1-2089673485", "_deleted": true},
>>>     {"_id": "expired_doc_2", "_rev": "1-2063452834", "_deleted": true}
>>>   ]
>>> }
>>> 
>>> 
>>> http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API#Modify_Multiple_Documents_With_a_Single_Request
for more info
>>> 
>>> 
>>> On Thursday, 30 August 2012 at 17:01, Stephan Bardubitzki wrote:
>>> 
>>>> Thanks Simon,
>>>> 
>>>> that's what I was thinking too, but have a problem to figure out how
>>>> bulk delete should be implemented. Do you have some advise or code
>>>> example on that?
>>>> 
>>>> Stephan
>>>> 
>>>> 
>>>> On 12-08-30 08:48 AM, Simon Metson wrote:
>>>>> You need to have a view keyed by expire_date and an external process
(some cron script, say) that queries that appropriately and makes the (bulk) delete of old
docs.
>>>>> Cheers
>>>>> Simon
>>>>> 
>>>>> 
>>>>> On Thursday, 30 August 2012 at 16:44, Stephan Bardubitzki wrote:
>>>>> 
>>>>>> I have a db with docs that have a field expire_date. I want to auto
>>>>>> delete those docs when expire_date has passed, but have no clue how
to
>>>>>> do that.
>>>>>> 
>>>>>> Any advise would be greatly appreciated.
>>>>>> 
>>>>>> Thanks,
>>>>>> Stephan
>>>>>> 
>>>>> 
>>>>> 
>>>>> --------------------------------
>>>>> Spam/Virus scanning by CanIt Pro
>>>>> 
>>>>> For more information see
>>>>> http://www.kgbinternet.com/SpamFilter.htm
>>>>> 
>>>>> To control your spam filter, log in at
>>>>> http://filter.kgbinternet.com
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> --------------------------------
>>> Spam/Virus scanning by CanIt Pro
>>> 
>>> For more information see
>>> http://www.kgbinternet.com/SpamFilter.htm
>>> 
>>> To control your spam filter, log in at
>>> http://filter.kgbinternet.com
>>> 
>> 
>> 
>> --------------------------------
>> Spam/Virus scanning by CanIt Pro
>> 
>> For more information see
>> http://www.kgbinternet.com/SpamFilter.htm
>> 
>> To control your spam filter, log in at
>> http://filter.kgbinternet.com
>> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message