incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Newson <robert.new...@gmail.com>
Subject Re: What am I doing wrong?
Date Fri, 20 Feb 2009 14:33:08 GMT
Compaction also makes the file non-sparse.

The 83Gb is probably not accurate, I suggest you compare what 'ls -lh'
shows versus what 'du -sh' does for the same file. I find the actual
consumed space is far, far less that 'ls' shows. CouchDB .couch files
are very sparse, large gaps of unwritten data, ostensibly to keep
btree and document items separate, but these 'holes' vanish after
compaction, even if you have zero updates and deletes.

B.

On Fri, Feb 20, 2009 at 2:03 PM, Pascal Borghino <pborghino@yahoo.fr> wrote:
> Hello guys,
> This is my first post, so I will take the opportunity to salute everyone. I
> started to play more with couchDB on monday.
>
> This is my box:
> Summary:    Dell 2950, 2 x Xeon L5320 1.86GHz, 4GB / 4GB 667MHz
> Disk:        sda (megaraid_sas0): 218GB (6%) RAID-1 == 6 x 73GB 15K SAS
> Seagate 15K.5
> OS:        RHEL AS 4 U4, Linux 2.6.9-42.ELsmp i686, 32-bit
>
> As you can see :
> {"db_name":"test","doc_count":7647662,"doc_del_count":0,"update_seq":7647662,"purge_seq":0,"compact_running":true,"disk_size":89081462811,"instance_start_time":"1234980644882221"}
>
> I have loaded over 7M documents. The original text file was 19Go, I splitted
> in 20000 docs pages and started 4 processes to load everything in parallel
> on couchdb.
> The whole process took over 16 hours so roughtly 130 doc /s.
>
> Everything went great, I am just really surprise of the result.
>
> From a 19Go text file, my test.couch file reached 83Go! How come? I am just
> really surprise of this result.
>
> I am currently compacting it... even  if 'Compaction rewrites the database
> file, removing outdated document revisions and deleted documents'... no
> document should be outdate neither deleted...
>
> 83G Feb 20 02:40 test.couch
> 15G Feb 20 13:52 test.couch.compact
>
> Am I doing something wrong?
> Thanks in advance
> Pascal
>
>
>
>

Mime
View raw message