couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Hsu <p...@abaca.com>
Subject view size extremely disk inefficient
Date Tue, 07 Jul 2009 18:56:58 GMT
I need help explaining why the sizes of my views are so large.

The emitted rows for the document have a key length of about 40 bytes (it's an array, if that
matters) and a view length of about 400 bytes (raw json).   However, I'm seeing over 2k/row
average over the view.  Factoring in the overhead of writing the btrees still doesn't really
make sense.

At 10M docs, I have almost 3.2k/message.  The views were generated at 3000 doc increments.
 At the beginning, with the first view generation, the view size was about 2MB.  This is about
800 bytes per row, which could be reasonable.

I measured the incremental size of the view after every 3000 rows were added to the view.
 By the time I'm at 30k rows, I'm seeing an increment in the view size of 4MB, which is over
1k/doc.  By the end of 10M messages, it's over a 10MB increment, which is over 3k/doc.

It may be interesting that a lot of my keys are identical.  Does that affect things?

This is with the 0.9.0 running on cent5 (64 bit).

Peter

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message