Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of robert.newson@gmail.com
 designates 209.85.162.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=CpYMEEvoYPFanV4iMf2LKvelGCkgg0BmFyNPtolmdhDSpFpZVZSFpbTqoDYsH57EkM
         lzGC7VDKGgkweOQN5HBH/DDfOijD09Bk0bOsC8notI7cZJadbDeQIz1JR3TQOCAd5z8G
         aFmTAcO256mDsu+MqUQV+nCtYVrXxl+jo7CHI=
MIME-Version: 1.0
In-Reply-To: <98EB4A72-76A5-4A9B-BD8C-6F04D26F3D88@mooseyard.com>
References: <499EB832.7040704@yahoo.fr>
	 <98EB4A72-76A5-4A9B-BD8C-6F04D26F3D88@mooseyard.com>
Date: Fri, 20 Feb 2009 16:23:32 +0000
Message-ID: <46aeb24f0902200823j77134ce2h4aeb84b1289d6863@mail.gmail.com>
Subject: Re: What am I doing wrong?
From: Robert Newson <robert.newson@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

I expect the b-tree wastage is minimal (though not zero).

I've wondered what happens on filesystems that don't support sparse
files, I assume they'd just be slower and use more disk space. Given
that the holes vanish after compaction, I suspected a bad calculation
in the code (couch_db.erl, I think), but I've not found it, it seems
to do the right thing. HFS+ doesn't support holes but I'm pretty sure
NTFS does.

Btw, it's mostly around attachments. If you add lots of documents but
no attachments, ls and df are in close agreement.

B.

On Fri, Feb 20, 2009 at 4:00 PM, Jens Alfke <jens@mooseyard.com> wrote:
>
> On Feb 20, 2009, at 6:03 AM, Pascal Borghino wrote:
>
>> I am currently compacting it... even  if 'Compaction rewrites the databa=
se
>> file, removing outdated document revisions and deleted documents'... no
>> document should be outdate neither deleted...
>
> In addition to the sparseness of the file, another reason for the size
> difference might be obsolete b-tree nodes. The file is append-only, so an=
y
> time a b-tree changes, the old nodes remain in the file. If you've done a
> large number of individual insertions, that space might be significant.
> (Probably not gigabytes, though.)
>
>
> robert.newson@gmail.com wrote:
>
>> I find the actual
>> consumed space is far, far less that 'ls' shows. CouchDB .couch files
>> are very sparse, large gaps of unwritten data, ostensibly to keep
>> btree and document items separate, but these 'holes' vanish after
>> compaction, even if you have zero updates and deletes.
>
> Hm. But not all filesystems support sparse files. HFS+, the Mac OS
> filesystem, doesn't. (Does NTFS?) Is there an option to suppress the gaps=
?
>
> =97Jens