incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elf <elf2...@gmail.com>
Subject Re: Incremental replication over unreliable link -- how granular is replication restart
Date Thu, 14 May 2009 14:10:14 GMT
Use rsync, Luke :)
It work over unreliable links, can resume aborted and partial files.

2009/5/14 Ben Cohen <cohen.ben@gmail.com>:
> Hi all --
>
> This is my first message to the list.  I've been watching it for a little
> while now and so far everything I read about the design of couchdb I like a
> lot!  Thanks so much for all the cool work!
>
> One of the uses I'm planning for couchdb involves replicating a database
> across a slow, unreliable link which will never become anything other than
> slow and unreliable.  I understand the replication is incremental and
> designed to 'pick up where it left off' in the case of replication
> interruption.  From the technical overview on the website:
>
>> The replication process is incremental. At the database level, replication
>> only examines documents updated since the last replication. Then for each
>> updated document, only fields and blobs that have changed are replicated
>> across the network. If replication fails at any step, due to network
>> problems or crash for example, the next replication restarts at the same
>> document where it left off.
>>
> I've got a question about this process.  Say you have a document to be
> replicated with a 1 megabyte attachment.  A replication process starts, half
> the doc is transferred successfully and then the connection dies.  Assuming
> no changes to the source doc, when the replication restarts will the
> transfer start from the beginning of the document or will it pick up
> somewhere within the doc?
>
> For my use case I have a slow link that will periodically come online for a
> certain fixed amount of time and initiate a replication.  If the replication
> isn't incremental 'within' a single document, then a document in the
> database above a certain size will for me, never make it across and I would
> imagine cause the replication to never make forward progress ...
>
> Does couchdb's replication magic avoid the issue for me and eventually
> transfer the document across my link?
>
> Thanks much,
> Ben Cohen



-- 
----------------
Best regards
Elf
mailto:elf2001@gmail.com

Mime
View raw message