couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Cohen <>
Subject Incremental replication over unreliable link -- how granular is replication restart
Date Thu, 14 May 2009 11:12:47 GMT
Hi all --

This is my first message to the list.  I've been watching it for a  
little while now and so far everything I read about the design of  
couchdb I like a lot!  Thanks so much for all the cool work!

One of the uses I'm planning for couchdb involves replicating a  
database across a slow, unreliable link which will never become  
anything other than slow and unreliable.  I understand the replication  
is incremental and designed to 'pick up where it left off' in the case  
of replication interruption.  From the technical overview on the  

> The replication process is incremental. At the database level,  
> replication only examines documents updated since the last  
> replication. Then for each updated document, only fields and blobs  
> that have changed are replicated across the network. If replication  
> fails at any step, due to network problems or crash for example, the  
> next replication restarts at the same document where it left off.
I've got a question about this process.  Say you have a document to be  
replicated with a 1 megabyte attachment.  A replication process  
starts, half the doc is transferred successfully and then the  
connection dies.  Assuming no changes to the source doc, when the  
replication restarts will the transfer start from the beginning of the  
document or will it pick up somewhere within the doc?

For my use case I have a slow link that will periodically come online  
for a certain fixed amount of time and initiate a replication.  If the  
replication isn't incremental 'within' a single document, then a  
document in the database above a certain size will for me, never make  
it across and I would imagine cause the replication to never make  
forward progress ...

Does couchdb's replication magic avoid the issue for me and eventually  
transfer the document across my link?

Thanks much,
Ben Cohen
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message