From user-return-14762-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Thu Feb 03 14:31:37 2011 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 76001 invoked from network); 3 Feb 2011 14:31:37 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 3 Feb 2011 14:31:37 -0000 Received: (qmail 92961 invoked by uid 500); 3 Feb 2011 14:31:35 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 92534 invoked by uid 500); 3 Feb 2011 14:31:31 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 92516 invoked by uid 99); 3 Feb 2011 14:31:30 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 14:31:30 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of zachary.zolton@gmail.com designates 74.125.82.180 as permitted sender) Received: from [74.125.82.180] (HELO mail-wy0-f180.google.com) (74.125.82.180) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Feb 2011 14:31:24 +0000 Received: by wyb28 with SMTP id 28so1192688wyb.11 for ; Thu, 03 Feb 2011 06:31:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type:content-transfer-encoding; bh=YsZ4ZAhhohhyE8xrAx4g1qG7tPGT2DX8S8iE5Iqth3o=; b=vVLMdCfGh7Bkbo2xIe+pt1Poj+cUMzTRHST5vkcrG5Vx3U4oflh7CVOd3YqEsmUxwW lcGAIFiEbb8DxtVThBPXsyvrh8vwjRvRbfplZ58VsuGQUnKM2UMfWyiJaUoB0Je1apxN 6wzE7T+wp+6Ohm/Jr0KOFkevh8O61zqTZtgDg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; b=tBG0xmpZN6UYBKpN5Br6TkKr2R6IcuB6gbdlYD2rkzxqG+jpxo8jbQ3qg5vdUSiRcI Pc7CwBgcQ6oyQ54gemXsjedqlBNa9wOUtcG9K860VzzXJz62G8RtjJ4bt1oYeIac2825 x6LsB9DXw3IcppIYhnu2OFhVcp+YHVNe7vfVc= Received: by 10.227.138.69 with SMTP id z5mr10832242wbt.174.1296743463982; Thu, 03 Feb 2011 06:31:03 -0800 (PST) MIME-Version: 1.0 Received: by 10.227.137.21 with HTTP; Thu, 3 Feb 2011 06:30:43 -0800 (PST) In-Reply-To: <003501cbc36b$67c1c420$37454c60$@vizrt.com> References: <000b01cbc1ed$902cb4d0$b0861e70$@vizrt.com> <001901cbc2bc$402ef310$c08cd930$@vizrt.com> <003501cbc36b$67c1c420$37454c60$@vizrt.com> From: Zachary Zolton Date: Thu, 3 Feb 2011 08:30:43 -0600 Message-ID: Subject: Re: Deleting Document To: user@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Whenever one creates or updates documents, those documents need to be re-indexed before the view queries will return any results. So if you delete 10,000 documents, it makes sense that a GET on your view might time out. What you've described sounds like the expected behavior of CouchDB. If you can set your HTTP library to never timeout, the view request will eventually respond, once the indexing has finished. You may also be able to reduce your total wait time by using a loop to delete a subset of documents and then hitting your view to keep it primed. Perhaps two or three thousand per iteration. This will help keep your view available. On Thu, Feb 3, 2011 at 12:27 AM, Panop S. wrote: > Hi, > > > > =A0 =A0 When I sent the bulk delete command to about =A010000 to couchDB = (1.0.0), > and this can be deleted succesfully. > > =A0 =A0 Unfortunately when I sent another get command it get operation ti= me > out. > > =A0 =A0 =A0I looks at that status it is something like " View Group Index= er : > Processed 5700 of 10000 changes (57%)". > > =A0 =A0 I have to wait patiently until the status is completed so I can g= et the > documents. > > > > =A0 =A0 Is =A0something wrong in couchdb ??? > > > > Thanks, > > > > > > > > -----Original Message----- > From: Zachary Zolton [mailto:zachary.zolton@gmail.com] > Sent: Wednesday, February 02, 2011 10:26 PM > To: user@couchdb.apache.org > Subject: Re: Deleting Document > > > > Bulk doc updates (of which deletion is a just a special case) has been kn= own > to perform well for thousands of documents at a time. > > > > I'd say querying your view with the limit parameter and performing bulk > updates in batches sounds like a good idea. If you anticipate needing to > work with 50,000 documents at a time you had spend some benchmarking. > Experiment with different batch size and perhaps share your results. > > > > -Zach > > > > On Wed, Feb 2, 2011 at 3:33 AM, Panop S. < > pas@vizrt.com> wrote: > >> Hi, > >> =A0 =A0 =A0 =A0 =A0 =A0Thanks for your suggestion, for bulk deletion let= 's say we > >> have > >> 50000 in one parent. > >> =A0 =A0 =A0 =A0 =A0 =A0 Is it good idea to load all at once with bulk de= letion ? > >> =A0 =A0 =A0 =A0 =A0 =A0 Or =A0I need reduce number to on view like limit= =3D1000 and > >> issue multiple requests. > >> > >> Big Thanks Again, > >> > >> > >> > >> > >> > >> -----Original Message----- > >> From: Zachary Zolton [mailto:zachary.zolton@gmail.com] > >> Sent: Tuesday, February 01, 2011 9:55 PM > >> To: user@couchdb.apache.org > >> Subject: Re: Deleting Document > >> > >> You could eliminate the recursion in your algorithms by storing an > >> array of ancestor ID in each of your documents. > >> > >> Then you could just write a by-ancestor view, like so: > >> > >> function(doc) { > >> =A0doc.ancestor_ids.forEach(function(id) { > >> =A0 =A0 =A0emit(id, null); > >> =A0}); > >> } > >> > >> When it's time to delete a document, you just query this view for all > >> docs that share the ancestor's ID and delete the matching documents. > >> Note that if you want to re-parent a doc you will also need to update > >> all its children in this scheme. > >> > >> For both of these operations you could consider using bulk doc updates: > >> =A0 http://is.gd/M9aNkg > >> > >> Depending on your project's requirements, you may benefit by using a > >> graph database instead. > >> > >> > >> Cheers, > >> > >> Zach > >> > >> On Tue, Feb 1, 2011 at 2:53 AM, Panop S. < > pas@vizrt.com> wrote: > >>> Hi, > >>> > >>> > >>> > >>> =A0 =A0 I have design a document in couchdb like a tree file structure = as > >>> > >>> > >>> > >>> =A0 =A0 =A01 > >>> > >>> =A0 =A0 =A0|--2 > >>> > >>> =A0 =A0 =A0| =A0 =A0|--4 > >>> > >>> =A0 =A0 =A0| > >>> > >>> =A0 =A0 =A0|--3 > >>> > >>> > >>> > >>> =A0 =A0 =A0Doc id : 2 > >>> > >>> =A0 =A0 =A0Parent: 1 > >>> > >>> > >>> > >>> =A0 =A0 =A0Doc id : 3 > >>> > >>> =A0 =A0 =A0Parent: 1 > >>> > >>> > >>> > >>> =A0 =A0 =A0Doc id : 4 > >>> > >>> =A0 =A0 =A0Parent: 1 > >>> > >>> > >>> > >>> =A0 =A0 =A0So when I query I emit it by parent Id . > >>> > >>> =A0 =A0 =A0Then when get =A0/id/1 , I will get docid =3D =A02 ,3 > >>> > >>> > >>> > >>> =A0 =A0 So if I would like to delete 1 require 2 steps in C# applicatio= n > >>> by using recursive call. > >>> > >>> > >>> > >>> 1. =A0 =A0 =A0 querying /id/1 =A0and issue get command id =3D 2 =A0and = issue > >>> delete command id =3D 2 > >>> > >>> 2. =A0 =A0 =A0 querying /id/2 =A0and issue get command id =3D 4 =A0and = issue > >>> delete command id =3D 4 > >>> > >>> > >>> > >>> =A0 =A0The problem is if there are a lot of documents it is slow down > >>> performance by recursive in C# code > >>> > >>> and query by loading huge documents =A0and the others client request > >>> cannot process. > >>> > >>> > >>> > >>> =A0 =A0As I can think of now for a lot of document by using =A0&limit= =3Dsome > >>> number to reduce a lot of tons > >>> > >>> Of document to be load. > >>> > >>> > >>> > >>> =A0 =A0 =A0 And also Can I write some operation to delete indexed id on > >>> javascript of couchdb ? if so, is this good idea ? > >>> > >>> =A0 =A0 =A0 =A0 =A0Any Idea ? > >>> > >>> > >>> > >>> Big Thanks, > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >> > >> > >> > > > >