Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.joseph.davis@gmail.com
 designates 209.85.220.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:from:date:message-id:subject:to
         :cc:content-type:content-transfer-encoding;
        b=ZFz3fFOjS72RyYPXs7K9UitmTyy1vj4+WemZWrbjx9FXQ24okcNUk9paoUC/ruVFdE
         IDqWV/WD1roJl6gxyJsyy4LUwvQtcoxA+8VMyEHrNoqqZgeg2uqQ4oMK2oqGWZ1wreRw
         9hJK5FXT4jwiKI8xmhFEe3SSIrpIgZngwqjqA=
MIME-Version: 1.0
In-Reply-To: <AANLkTi=r8HiQDY2ay6BVP5os+FvAvQSvwv9XcQGgzK4r@mail.gmail.com>
References: <AANLkTi=r8HiQDY2ay6BVP5os+FvAvQSvwv9XcQGgzK4r@mail.gmail.com>
From: Paul Davis <paul.joseph.davis@gmail.com>
Date: Tue, 15 Mar 2011 14:41:45 -0400
Message-ID: <AANLkTik1abAZJD0uMwYU5Yo2D1ofgyMG+Wcy4A6DNWAy@mail.gmail.com>
Subject: Re: Some CouchDB internals questions?
To: user@couchdb.apache.org
Cc: Zdravko Gligic <zgligic@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Tue, Mar 15, 2011 at 2:37 PM, Zdravko Gligic <zgligic@gmail.com> wrote:
> Folks,
>
> I created a sample proof of concept app and it created more questions
> than it answered with regard to CouchDB and resource usage. =A0From the
> best that I can tell, some 1,500 docs which are about 500 characters
> in size (in terms of readable text) to maybe 1k at most are being
> bloated to an average of 10k per doc - with just 2 'plain' views and a
> 3rd view that re-emits the doc as well.
>
> So, is there any literature around that explains at least a bit some
> of the CouchDB internals, in terms of at least disk usage. =A0How much
> might each byte end up taking to store (with whatever built-in
> overhead) in a doc and what about views and indexing? Are there any
> even very generalized formulas like multiply all bytes in index by 2
> and add 10 bytes for each filed/attribute in index and then add 1k if
> you are building indices on a cloudy day ;?)
>
> Thanks for your help,
> Zdravko
>

Have you compacted your db and views?

There's unfortunately no direct way to calculate a upper threshold, it
really depends on your method for inserting as well as how often you
compact. This is due to the tail append storage which will orphan data
in the file as it writes new records to the various internal data
structures.