couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Marino <mmar...@gmail.com>
Subject Re: Replication without IP connection
Date Mon, 27 Jan 2014 18:52:28 GMT
If you absolutely must use the network transfer, and are looking to do
something special to the data between points (ie before the data
traverses the network), netcat is an excellent tool for patching
together protocols/operating on the data. You might find something
there which could help you along the lines of what Jens suggested.

> Am 27.01.2014 um 19:18 schrieb Jens Alfke <jens@couchbase.com>:
>
>
>> On Jan 27, 2014, at 10:04 AM, Benoit Chesneau <bchesneau@gmail.com> wrote:
>>
>> How would you do your file transfer? Other than that you could listen on
>> couchdb changes and rsync your databases files between nodes.
>
> Nope, rsync is a highly interactive protocol that runs over IP. It's also not very efficient
compared to delta-coding, especially when you toss an entire database file at it instead of
just cherry-picking the JSON bodies that changed.
>
> If it helps, think of it as though the _only_ tool you could use to communicate were
scp. And you're running it over a really crappy 2G GPRS/EDGE cell connection.
>
> I tried to look up info on the available bit-rate, but apparently "HF Radio" is a very
generic term. Aaron, can you share specifics of the technology you're using and its bandwidth/latency
characteristics?
>
> Here's the beginnings of an approach:
> GET /db/_changes?conflicts=true&style=all_docs&include_docs=true&attachments=true&since=XXX
> where XXX is the db sequence number from the last time you ran a push.
> Then compress the hell out of the response (it's full of JSON so it's going to be very
redundant) and send it across.
>
> On the receiving end you'll probably have to write custom code to unpack the individual
revisions out of the feed and apply them to the database.
>
> --Jens

Mime
View raw message