couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: What am I doing wrong?
Date Fri, 20 Feb 2009 16:40:49 GMT

On 20 Feb 2009, at 17:23, Robert Newson wrote:

> I expect the b-tree wastage is minimal (though not zero).

actually not, only after compaction.

Cheers
Jan
--


> I've wondered what happens on filesystems that don't support sparse
> files, I assume they'd just be slower and use more disk space. Given
> that the holes vanish after compaction, I suspected a bad calculation
> in the code (couch_db.erl, I think), but I've not found it, it seems
> to do the right thing. HFS+ doesn't support holes but I'm pretty sure
> NTFS does.
>
> Btw, it's mostly around attachments. If you add lots of documents but
> no attachments, ls and df are in close agreement.
>
> B.
>
> On Fri, Feb 20, 2009 at 4:00 PM, Jens Alfke <jens@mooseyard.com>  
> wrote:
>>
>> On Feb 20, 2009, at 6:03 AM, Pascal Borghino wrote:
>>
>>> I am currently compacting it... even  if 'Compaction rewrites the  
>>> database
>>> file, removing outdated document revisions and deleted  
>>> documents'... no
>>> document should be outdate neither deleted...
>>
>> In addition to the sparseness of the file, another reason for the  
>> size
>> difference might be obsolete b-tree nodes. The file is append-only,  
>> so any
>> time a b-tree changes, the old nodes remain in the file. If you've  
>> done a
>> large number of individual insertions, that space might be  
>> significant.
>> (Probably not gigabytes, though.)
>>
>>
>> robert.newson@gmail.com wrote:
>>
>>> I find the actual
>>> consumed space is far, far less that 'ls' shows. CouchDB .couch  
>>> files
>>> are very sparse, large gaps of unwritten data, ostensibly to keep
>>> btree and document items separate, but these 'holes' vanish after
>>> compaction, even if you have zero updates and deletes.
>>
>> Hm. But not all filesystems support sparse files. HFS+, the Mac OS
>> filesystem, doesn't. (Does NTFS?) Is there an option to suppress  
>> the gaps?
>>
>> —Jens
>


Mime
View raw message