Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of adam.kocoloski@gmail.com
 designates 209.85.216.190 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=sender:message-id:from:to:in-reply-to:content-type
         :content-transfer-encoding:mime-version:subject:date:references
         :x-mailer;
        b=AeE1GtZCgMq6P3ydWS2BwrC3K0wkcEfVE4scgRvYOEJa7ir48mevzjjPQD8QSwpCcn
         YfDaaCMQe253QgXh5oHx3nBVKi9lvMVj6JJEvWgvjIOkUfqKqW41foaYumVNCJcsSXsB
         /6zTG24kvS5djpGYGNpFXE3Vi9KbG/EK7Lres=
Sender: Adam Kocoloski <adam.kocoloski@gmail.com>
Message-Id: <830466B8-361B-47A4-89AE-1990FD4CBBD8@apache.org>
From: Adam Kocoloski <kocolosk@apache.org>
To: user@couchdb.apache.org
In-Reply-To: <22E9DC658696E640A074D8B363795BE62D6C5C1C54@mail1.ABACA.local>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (Apple Message framework v935.3)
Subject: Re: view size extremely disk inefficient
Date: Tue, 7 Jul 2009 15:15:42 -0400
References: <22E9DC658696E640A074D8B363795BE62D6C5C1C54@mail1.ABACA.local>

On Jul 7, 2009, at 2:56 PM, Peter Hsu wrote:

> I need help explaining why the sizes of my views are so large.
>
> The emitted rows for the document have a key length of about 40  
> bytes (it's an array, if that matters) and a view length of about  
> 400 bytes (raw json).   However, I'm seeing over 2k/row average over  
> the view.  Factoring in the overhead of writing the btrees still  
> doesn't really make sense.
>
> At 10M docs, I have almost 3.2k/message.  The views were generated  
> at 3000 doc increments.  At the beginning, with the first view  
> generation, the view size was about 2MB.  This is about 800 bytes  
> per row, which could be reasonable.
>
> I measured the incremental size of the view after every 3000 rows  
> were added to the view.  By the time I'm at 30k rows, I'm seeing an  
> increment in the view size of 4MB, which is over 1k/doc.  By the end  
> of 10M messages, it's over a 10MB increment, which is over 3k/doc.
>
> It may be interesting that a lot of my keys are identical.  Does  
> that affect things?
>
> This is with the 0.9.0 running on cent5 (64 bit).
>
> Peter

Hi Peter, have you tried compacting the view?

POST /dbname/_compact/designname

Best, Adam