incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Couchdb 0.11 replication issue
Date Thu, 22 Apr 2010 11:19:12 GMT
Actually, it's hard to make an a priori calculation about the size of the target DB.  It's
true that replication is like compaction in that it removes old MVCC revisions.  On the other
hand, file size on disk is also driven by the size of write batches.  If the target is fast
relative to the source you many end in a situation where the the replication is writing one
document at a time on the target, which is a worst-case scenario for file sizes.

Compaction uses constant-size batches, so it's not subject to this variability.  Compacting
the target DB will restore the smaller file size.  Best,

Adam

On Apr 22, 2010, at 5:35 AM, Filipe David Manana wrote:

> Paul,
> 
> Usually it's the other way around. The replica DB usually stays smaller than
> the source DB (unless the target DB was already populated). This is so
> because doing a replica is similar to a DB compaction in many ways.
> 
> On Thu, Apr 22, 2010 at 6:26 AM, Paul Bonser <pib@paulbonser.com> wrote:
> 
>> I think that's pretty typical behavior, since it's append-only
>> re-building any views you have. Try compacting the views and database
>> and see if it's still bigger. My guess is that it won't be.
>> 
>> I have a DB which did about the same thing. I replicated from one
>> machine to another, and it grew from 209MB to a little less than a GB,
>> then after compacting, it's back down to 209MB. Though in my case the
>> same thing happened during my initial insert of the data, too, because
>> I was reading from the DB as I was doing the inserts (and thus the
>> views were being rebuilt, ballooning the file).
>> 
>> On Tue, Apr 20, 2010 at 2:08 PM, Bharat Bharat <Bharat.Bharat@citrix.com>
>> wrote:
>>> 
>>> Hi, I am seeing a very bizzare issue with couchdb 0.11 replication.
>>> 
>>> When I try to replicate a database from my local machine (mac OSX) to a
>>> remote machine( on a different network), it replicates all documents but
>>> size of the document on the remote machine goes up around 10 times and it
>>> eventually fails.
>>> 
>>> eg: Host machine: 450 docs, ~45 MB
>>>    Target machine after replication: 450 docs, ~ 400 MB
>>> 
>>> Interestingly, the other way works fine. Ie, if I try to pull this
>> database
>>> from host machine to this remote machine, it replicates fine.
>>> 
>>> Has anyone encountered this or something similar? I dont have big
>>> attachments.
>>> 
>>> Thanks very much!
>> 
>> 
>> 
>> --
>> Paul Bonser
>> http://probablyprogramming.com
>> 
> 
> 
> 
> -- 
> Filipe David Manana,
> fdmanana@gmail.com
> 
> "Reasonable men adapt themselves to the world.
> Unreasonable men adapt the world to themselves.
> That's why all progress depends on unreasonable men."


Mime
View raw message