Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of snbartell@gmail.com
 designates 209.85.160.52 as permitted sender)
From: stephen bartell <snbartell@gmail.com>
Mime-Version: 1.0 (Apple Message framework v1257)
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_E1BC1D35-6C58-4854-9A87-D792F98DFC8B"
Subject: Re: auto delete docs
Date: Thu, 30 Aug 2012 10:26:10 -0700
In-Reply-To: <503F9BBD.6010603@bardubitzki.com>
To: user@couchdb.apache.org
References: <503F8A7A.10503@bardubitzki.com>
 <DE875974BB0449A3A9D7662B143D9CB6@cloudant.com>
 <503F8E4C.2060600@bardubitzki.com>
 <820406CE2A8D4A7CA9E538821094CB52@cloudant.com>
 <503F9510.2050002@bardubitzki.com> <503F9BBD.6010603@bardubitzki.com>
Message-Id: <03FFF0D9-87DB-4884-821B-DA9D9B5CB386@gmail.com>

--Apple-Mail=_E1BC1D35-6C58-4854-9A87-D792F98DFC8B
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

What Simon said.  Im doing this in production.  I have a python script =
set up as a cron job which queries the expire_time view.  Docs that get =
returned get deleted. Its a real small, simple script and works like a =
charm.

Im pretty sure you know this, but make sure to page through your view =
results.  If your database hasn't been 'cleaned out' for a while and the =
database is huge, you're going to get quite a massive response.  I once =
made the mistake of not paging my results in a node program.  V8's mem =
limit is 1gb on a 64 bit machine, so once this limit was exceeded, the =
program would crash and never complete its job.  You could see where =
this leads if that program is responsible for getting rid of old docs =
and thus keeping database size under control.

Stephen Bartell

Look ahead, understand the shift, and imagine the right solution five =
years from now. Then do it!
-baekdal

On Aug 30, 2012, at 9:58 AM, Stephan Bardubitzki wrote:

> Okay, the link works now. Must have been an issue at Apache.
>=20
> On 12-08-30 09:30 AM, Stephan Bardubitzki wrote:
>> @Simon:
>>=20
>> Thanks, again. Could you please double check the link, I can't open =
it.
>>=20
>> @Matthieu
>>=20
>> Unfortunately, I'm not familiar with Erlang. I need to do this task =
on node.js with nano (https://github.com/dscape/nano).
>>=20
>>=20
>> On 12-08-30 09:03 AM, Simon Metson wrote:
>>> Sure, you can use _bulk_docs and send a json doc like:
>>>=20
>>> {
>>>   "docs": [
>>>     {"_id": "expired_doc_0", "_rev": "1-62657917", "_deleted": =
true},
>>>     {"_id": "expired_doc_1", "_rev": "1-2089673485", "_deleted": =
true},
>>>     {"_id": "expired_doc_2", "_rev": "1-2063452834", "_deleted": =
true}
>>>   ]
>>> }
>>>=20
>>>=20
>>> =
http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API#Modify_Multiple_Docu=
ments_With_a_Single_Request for more info
>>>=20
>>>=20
>>> On Thursday, 30 August 2012 at 17:01, Stephan Bardubitzki wrote:
>>>=20
>>>> Thanks Simon,
>>>>=20
>>>> that's what I was thinking too, but have a problem to figure out =
how
>>>> bulk delete should be implemented. Do you have some advise or code
>>>> example on that?
>>>>=20
>>>> Stephan
>>>>=20
>>>>=20
>>>> On 12-08-30 08:48 AM, Simon Metson wrote:
>>>>> You need to have a view keyed by expire_date and an external =
process (some cron script, say) that queries that appropriately and =
makes the (bulk) delete of old docs.
>>>>> Cheers
>>>>> Simon
>>>>>=20
>>>>>=20
>>>>> On Thursday, 30 August 2012 at 16:44, Stephan Bardubitzki wrote:
>>>>>=20
>>>>>> I have a db with docs that have a field expire_date. I want to =
auto
>>>>>> delete those docs when expire_date has passed, but have no clue =
how to
>>>>>> do that.
>>>>>>=20
>>>>>> Any advise would be greatly appreciated.
>>>>>>=20
>>>>>> Thanks,
>>>>>> Stephan
>>>>>>=20
>>>>>=20
>>>>>=20
>>>>> --------------------------------
>>>>> Spam/Virus scanning by CanIt Pro
>>>>>=20
>>>>> For more information see
>>>>> http://www.kgbinternet.com/SpamFilter.htm
>>>>>=20
>>>>> To control your spam filter, log in at
>>>>> http://filter.kgbinternet.com
>>>>>=20
>>>>=20
>>>>=20
>>>=20
>>>=20
>>> --------------------------------
>>> Spam/Virus scanning by CanIt Pro
>>>=20
>>> For more information see
>>> http://www.kgbinternet.com/SpamFilter.htm
>>>=20
>>> To control your spam filter, log in at
>>> http://filter.kgbinternet.com
>>>=20
>>=20
>>=20
>> --------------------------------
>> Spam/Virus scanning by CanIt Pro
>>=20
>> For more information see
>> http://www.kgbinternet.com/SpamFilter.htm
>>=20
>> To control your spam filter, log in at
>> http://filter.kgbinternet.com
>>=20
>=20


--Apple-Mail=_E1BC1D35-6C58-4854-9A87-D792F98DFC8B--