couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zachary Zolton <zachary.zol...@gmail.com>
Subject Re: Deleting Document
Date Thu, 03 Feb 2011 14:30:43 GMT
Whenever one creates or updates documents, those documents need to be
re-indexed before the view queries will return any results. So if you
delete 10,000 documents, it makes sense that a GET on your view might
time out. What you've described sounds like the expected behavior of
CouchDB.

If you can set your HTTP library to never timeout, the view request
will eventually respond, once the indexing has finished.

You may also be able to reduce your total wait time by using a loop to
delete a subset of documents and then hitting your view to keep it
primed. Perhaps two or three thousand per iteration. This will help
keep your view available.


On Thu, Feb 3, 2011 at 12:27 AM, Panop S. <pas@vizrt.com> wrote:
> Hi,
>
>
>
>     When I sent the bulk delete command to about  10000 to couchDB (1.0.0),
> and this can be deleted succesfully.
>
>     Unfortunately when I sent another get command it get operation time
> out.
>
>      I looks at that status it is something like " View Group Indexer :
> Processed 5700 of 10000 changes (57%)".
>
>     I have to wait patiently until the status is completed so I can get the
> documents.
>
>
>
>     Is  something wrong in couchdb ???
>
>
>
> Thanks,
>
>
>
>
>
>
>
> -----Original Message-----
> From: Zachary Zolton [mailto:zachary.zolton@gmail.com]
> Sent: Wednesday, February 02, 2011 10:26 PM
> To: user@couchdb.apache.org
> Subject: Re: Deleting Document
>
>
>
> Bulk doc updates (of which deletion is a just a special case) has been known
> to perform well for thousands of documents at a time.
>
>
>
> I'd say querying your view with the limit parameter and performing bulk
> updates in batches sounds like a good idea. If you anticipate needing to
> work with 50,000 documents at a time you had spend some benchmarking.
> Experiment with different batch size and perhaps share your results.
>
>
>
> -Zach
>
>
>
> On Wed, Feb 2, 2011 at 3:33 AM, Panop S. < <mailto:pas@vizrt.com>
> pas@vizrt.com> wrote:
>
>> Hi,
>
>>            Thanks for your suggestion, for bulk deletion let's say we
>
>> have
>
>> 50000 in one parent.
>
>>             Is it good idea to load all at once with bulk deletion ?
>
>>             Or  I need reduce number to on view like limit=1000 and
>
>> issue multiple requests.
>
>>
>
>> Big Thanks Again,
>
>>
>
>>
>
>>
>
>>
>
>>
>
>> -----Original Message-----
>
>> From: Zachary Zolton [mailto:zachary.zolton@gmail.com]
>
>> Sent: Tuesday, February 01, 2011 9:55 PM
>
>> To: user@couchdb.apache.org
>
>> Subject: Re: Deleting Document
>
>>
>
>> You could eliminate the recursion in your algorithms by storing an
>
>> array of ancestor ID in each of your documents.
>
>>
>
>> Then you could just write a by-ancestor view, like so:
>
>>
>
>> function(doc) {
>
>>  doc.ancestor_ids.forEach(function(id) {
>
>>      emit(id, null);
>
>>  });
>
>> }
>
>>
>
>> When it's time to delete a document, you just query this view for all
>
>> docs that share the ancestor's ID and delete the matching documents.
>
>> Note that if you want to re-parent a doc you will also need to update
>
>> all its children in this scheme.
>
>>
>
>> For both of these operations you could consider using bulk doc updates:
>
>>  <http://is.gd/M9aNkg> http://is.gd/M9aNkg
>
>>
>
>> Depending on your project's requirements, you may benefit by using a
>
>> graph database instead.
>
>>
>
>>
>
>> Cheers,
>
>>
>
>> Zach
>
>>
>
>> On Tue, Feb 1, 2011 at 2:53 AM, Panop S. < <mailto:pas@vizrt.com>
> pas@vizrt.com> wrote:
>
>>> Hi,
>
>>>
>
>>>
>
>>>
>
>>>     I have design a document in couchdb like a tree file structure as
>
>>>
>
>>>
>
>>>
>
>>>      1
>
>>>
>
>>>      |--2
>
>>>
>
>>>      |    |--4
>
>>>
>
>>>      |
>
>>>
>
>>>      |--3
>
>>>
>
>>>
>
>>>
>
>>>      Doc id : 2
>
>>>
>
>>>      Parent: 1
>
>>>
>
>>>
>
>>>
>
>>>      Doc id : 3
>
>>>
>
>>>      Parent: 1
>
>>>
>
>>>
>
>>>
>
>>>      Doc id : 4
>
>>>
>
>>>      Parent: 1
>
>>>
>
>>>
>
>>>
>
>>>      So when I query I emit it by parent Id .
>
>>>
>
>>>      Then when get  /id/1 , I will get docid =  2 ,3
>
>>>
>
>>>
>
>>>
>
>>>     So if I would like to delete 1 require 2 steps in C# application
>
>>> by using recursive call.
>
>>>
>
>>>
>
>>>
>
>>> 1.       querying /id/1  and issue get command id = 2  and issue
>
>>> delete command id = 2
>
>>>
>
>>> 2.       querying /id/2  and issue get command id = 4  and issue
>
>>> delete command id = 4
>
>>>
>
>>>
>
>>>
>
>>>    The problem is if there are a lot of documents it is slow down
>
>>> performance by recursive in C# code
>
>>>
>
>>> and query by loading huge documents  and the others client request
>
>>> cannot process.
>
>>>
>
>>>
>
>>>
>
>>>    As I can think of now for a lot of document by using  &limit=some
>
>>> number to reduce a lot of tons
>
>>>
>
>>> Of document to be load.
>
>>>
>
>>>
>
>>>
>
>>>       And also Can I write some operation to delete indexed id on
>
>>> javascript of couchdb ? if so, is this good idea ?
>
>>>
>
>>>          Any Idea ?
>
>>>
>
>>>
>
>>>
>
>>> Big Thanks,
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>>
>
>>
>
>>
>
>>
>
>
>
>

Mime
View raw message