Mailing-List: contact couchdb-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: couchdb-user@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of damienkatz@gmail.com
 designates 64.233.184.224 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:from:to:in-reply-to:content-type
         :content-transfer-encoding:mime-version:subject:date:references
         :x-mailer;
        b=EK8NkDN4hHEuwkW3VkDqhpAbSaHmORie2lz0GLxTHMb6s8UMBNAgx5+XmDnRdenMCH
         NRykKfzUm7eqleyyU8Ni68IhIRfVFYVedXbG1rUkERLtnWio9/xvUDzmixyv02qEm7Qi
         Zj+/rzqXPD2MJ6sWI0Gwd1gJC5lx5g8Jnimeg=
Message-Id: <D78EFCA8-55EA-40E0-BF21-D99984E908D1@gmail.com>
From: Damien Katz <damienkatz@gmail.com>
To: couchdb-user@incubator.apache.org
In-Reply-To: <888cd9180806201351u1373f465r606d9c37d1903a19@mail.gmail.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v924)
Subject: Re: view index build time
Date: Sat, 21 Jun 2008 08:39:15 -0400
References: <888cd9180806201345k423a4dc8j64a881f351bd60bd@mail.gmail.com>
 <e282921e0806201349g6026df96j6cd40f5c70154766@mail.gmail.com>
 <888cd9180806201351u1373f465r606d9c37d1903a19@mail.gmail.com>

Part of the problem is you are storing copies of the documents into  
the btree. If the documents are big, it takes longer to compute on  
them, and if the results (emit(...)) are big or numerous, then you'll  
be spending most of your time in I/O.

My advice is to not emit the document into the view, and if you can,  
get the documents smaller in general. If the data can stored as an  
binary attachment, then that too will give you a performance  
improvement.

-Damien

On Jun 20, 2008, at 4:51 PM, Brad King wrote:

> Thanks, yes its currently at 357M and growing!
>
> On Fri, Jun 20, 2008 at 4:49 PM, Chris Anderson <jchris@grabb.it>  
> wrote:
>> Brad,
>>
>> You can look at
>>
>> ls -lha /usr/local/var/lib/couchdb/.my-dbname_design/
>>
>> to see the view size growing...
>>
>> It won't tell you when it's done but it will give you hope that the
>> progress is happening.
>>
>> Chris
>>
>> On Fri, Jun 20, 2008 at 1:45 PM, Brad King <brking@gmail.com> wrote:
>>> I have about 350K documents in a database. typically around 5K  
>>> each. I
>>> created and saved a view which simply looks at one field in the
>>> document. I called the view for the first time with a key that  
>>> should
>>> only match one document, and its been awaiting a response for  
>>> about 45
>>> minutes now.
>>>
>>> {
>>>  "sku": {
>>>      "map": "function(doc) { emit(doc.entityobject.SKU, doc); }"
>>>  }
>>> }
>>>
>>> Is this typical, or is there some optimizing to be done on either my
>>> view or the server? I'm also running on a VM so this may have some
>>> effects, but smaller databases seem to be performing pretty well.
>>> Insert times to set this up were actually really good I thought, at
>>> 4000 to 5000 documents per minute running from my laptop.
>>>
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchris.mfdz.com
>>