couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: view size extremely disk inefficient
Date Tue, 07 Jul 2009 21:10:05 GMT
Oops, I guess it's not.  Turns out I checked in that code a week after  
0.9.0 was released.  Sorry for the confusion.

As Chris said, if trunk is an option you might try that.  Best,

Adam

On Jul 7, 2009, at 4:52 PM, Peter Hsu wrote:

> Is that available in 0.9.0?
>
> I tried a POST but I get a 405 error.
>
> [Tue, 07 Jul 2009 13:46:08 GMT] [info] [<0.22689.306>] 172.16.80.64  
> - - 'GET' /mydb/_compact/meta_in 404
>
> [Tue, 07 Jul 2009 13:47:43 GMT] [info] [<0.22706.306>] 172.16.80.64  
> - - 'POST' /mydb/_compact/meta_in 405
>
> -----Original Message-----
> From: Adam Kocoloski [mailto:kocolosk@apache.org]
> Sent: Tuesday, July 07, 2009 12:16 PM
> To: user@couchdb.apache.org
> Subject: Re: view size extremely disk inefficient
>
> On Jul 7, 2009, at 2:56 PM, Peter Hsu wrote:
>
>> I need help explaining why the sizes of my views are so large.
>>
>> The emitted rows for the document have a key length of about 40
>> bytes (it's an array, if that matters) and a view length of about
>> 400 bytes (raw json).   However, I'm seeing over 2k/row average over
>> the view.  Factoring in the overhead of writing the btrees still
>> doesn't really make sense.
>>
>> At 10M docs, I have almost 3.2k/message.  The views were generated
>> at 3000 doc increments.  At the beginning, with the first view
>> generation, the view size was about 2MB.  This is about 800 bytes
>> per row, which could be reasonable.
>>
>> I measured the incremental size of the view after every 3000 rows
>> were added to the view.  By the time I'm at 30k rows, I'm seeing an
>> increment in the view size of 4MB, which is over 1k/doc.  By the end
>> of 10M messages, it's over a 10MB increment, which is over 3k/doc.
>>
>> It may be interesting that a lot of my keys are identical.  Does
>> that affect things?
>>
>> This is with the 0.9.0 running on cent5 (64 bit).
>>
>> Peter
>
> Hi Peter, have you tried compacting the view?
>
> POST /dbname/_compact/designname
>
> Best, Adam


Mime
View raw message