incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filipe David Manana <fdman...@apache.org>
Subject Re: Couch Replication: slave on-disk size bloat
Date Fri, 08 Apr 2011 15:38:55 GMT
On Fri, Apr 8, 2011 at 4:31 PM,  <steve.webb@beatport.com> wrote:
> I'm not sure what's in the databases.  Why?  Would large attachments explain
> this in some way?

Yes.
The current replicator in 1.0.x and 1.1 (yet to be released), when
replicating a new revision of a document with attachments, it
replicates all the attachments, not just the attachments missing on
the target. So you might end up with the same attachments several
times in the target database file (compaction will fix this of
course).

The new replicator, in trunk only, doesn't have this behaviour - it
replicates only the missing attachments.

>
> - Steve
>
> On Fri, 8 Apr 2011, Filipe David Manana wrote:
>
>> Do you have many and/or large attachments?
>>
>> On Thu, Apr 7, 2011 at 10:24 PM,  <steve.webb@beatport.com> wrote:
>>>
>>> Hello there.
>>>
>>> New couch admin here.
>>>
>>> I've got two couchdb machines that are not being written to.
>>>
>>> I cleaned off the slave, then replicated and compacted the DBs on the
>>> slave
>>> and the slave DB size on-disk is *way* more than on the master.
>>>
>>> Also, we've found that when the slaves are being actively written
>>> (replicated) to, over time, they grow > 10x the size of the master's file
>>> size.
>>>
>>> Master:
>>>
>>> -rw-r--r--  1 couchdb couchdb      20565 Apr  7 12:02 api-keys.couch
>>> -rw-r--r--  1 couchdb couchdb 2414678118 Apr  7 11:41
>>> similar-tracks.couch
>>> -rw-r--r--  1 couchdb couchdb  851714150 Apr  7 11:52 sitemap.couch
>>> drwxr-xr-x  2 couchdb couchdb       4096 Feb 15 01:10 .sitemap_design
>>> -rw-r--r--  1 couchdb couchdb   76939362 Apr  7 11:27 top-downloads.couch
>>> -rw-r--r--  1 couchdb couchdb  268513382 Apr  7 11:30
>>> users-also-bought.couch
>>>
>>> Slave (after delete, replication and compaction):
>>>
>>> -rw-r--r--  1 couchdb couchdb       8280 Apr  7 12:02 api-keys.couch
>>> -rw-r--r--  1 couchdb couchdb 5261164653 Apr  7 14:16
>>> similar-tracks.couch
>>> -rw-r--r--  1 couchdb couchdb  860319850 Apr  7 13:58 sitemap.couch
>>> -rw-r--r--  1 couchdb couchdb  122777700 Apr  7 13:50 top-downloads.couch
>>> -rw-r--r--  1 couchdb couchdb  285810794 Apr  7 13:52
>>> users-also-bought.couch
>>>
>>> Q1: Is this normal?
>>> Q2: Is there a way to fix this or turn off whatever is turned on that
>>> causes
>>> this?
>>>
>>> - Steve Webb
>>>
>>> --
>>> Steve Webb | System Administrator
>>> Beatport | Play With Music
>>> ------------------------------------------
>>> 2399 Blake Street, Suite 170
>>> Denver, Colorado USA 80205
>>> tel: +1.720.932.9103
>>> fax: +1.720.932.9104
>>> noc: +1.303.565.2710
>>> mobile: +1.303.564.4269
>>>
>>
>>
>>
>>
>
> --
> Steve Webb | System Administrator
> Beatport | Play With Music
> ------------------------------------------
> 2399 Blake Street, Suite 170
> Denver, Colorado USA 80205
> tel: +1.720.932.9103
> fax: +1.720.932.9104
> noc: +1.303.565.2710
> mobile: +1.303.564.4269



-- 
Filipe David Manana,
fdmanana@gmail.com, fdmanana@apache.org

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Mime
View raw message