couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pascal Borghino <pborgh...@yahoo.fr>
Subject Re: What am I doing wrong?
Date Fri, 20 Feb 2009 16:42:13 GMT
Hi there, I do not have attachments...

$ ls -lh
-rw-r--r--  1 root root  83G Feb 20 02:40 test.couch
-rw-r--r--  1 root root  23G Feb 20 16:33 test.couch.compact

$ du -sh
107G    .

still... from 19Go to 83Go... huge difference.
P.


Robert Newson a écrit :
> I expect the b-tree wastage is minimal (though not zero).
>
> I've wondered what happens on filesystems that don't support sparse
> files, I assume they'd just be slower and use more disk space. Given
> that the holes vanish after compaction, I suspected a bad calculation
> in the code (couch_db.erl, I think), but I've not found it, it seems
> to do the right thing. HFS+ doesn't support holes but I'm pretty sure
> NTFS does.
>
> Btw, it's mostly around attachments. If you add lots of documents but
> no attachments, ls and df are in close agreement.
>
> B.
>
> On Fri, Feb 20, 2009 at 4:00 PM, Jens Alfke <jens@mooseyard.com> wrote:
>   
>> On Feb 20, 2009, at 6:03 AM, Pascal Borghino wrote:
>>
>>     
>>> I am currently compacting it... even  if 'Compaction rewrites the database
>>> file, removing outdated document revisions and deleted documents'... no
>>> document should be outdate neither deleted...
>>>       
>> In addition to the sparseness of the file, another reason for the size
>> difference might be obsolete b-tree nodes. The file is append-only, so any
>> time a b-tree changes, the old nodes remain in the file. If you've done a
>> large number of individual insertions, that space might be significant.
>> (Probably not gigabytes, though.)
>>
>>
>> robert.newson@gmail.com wrote:
>>
>>     
>>> I find the actual
>>> consumed space is far, far less that 'ls' shows. CouchDB .couch files
>>> are very sparse, large gaps of unwritten data, ostensibly to keep
>>> btree and document items separate, but these 'holes' vanish after
>>> compaction, even if you have zero updates and deletes.
>>>       
>> Hm. But not all filesystems support sparse files. HFS+, the Mac OS
>> filesystem, doesn't. (Does NTFS?) Is there an option to suppress the gaps?
>>
>> —Jens
>>     
>
>   



Mime
View raw message