couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Marino <mmar...@gmail.com>
Subject Re: Replication without IP connection
Date Mon, 27 Jan 2014 17:42:42 GMT
This is an interesting problem.  Aaron, can you say something more about
what sorts of data transfer you have at all?  (USB stick/hard drive
transfer through the mail?)  Do you only do one-way replication?  We have
had to do such data transfers for our experiment, but not for isolated DBs.


During any transfer process, is it possible for the system itself to be
down i.e. not writeable for an extended period of time?  How are read/write
behaviours for data shared between central and remote nodes?

What about:

Remote system A, Central System B

1.  Make A read-only, copy database file to intermediate A_1

2. "Transfer" A_1 to system accessible to B.

3. Replicate A_1 to B.  (Two-way replication should then also change A_1,
if that's needed)

4. "Transfer" A_1 back to system accessible to A.

5. Make A only writable for replication from A_1, replicate A_1 to A.

6. Make A writable.

It's possible you could relax making A read-only if, for example, if system
B is only changing data that system A reads.

Cheers,
Mike




On Mon, Jan 27, 2014 at 6:15 PM, Aaron Huslage <huslage@gmail.com> wrote:

> Thanks Scott. Copying datafiles isn't really an option here. The files are
> just too large for the bandwidth available. And datafiles don't really
> compress much.
>
> The base system without any data is about 3mb. Copying a changed 2.6mb ddoc
> over the HF link would take most of a day, for instance.
>
>
> On Mon, Jan 27, 2014 at 12:00 PM, Scott Weber <scotty2541@sbcglobal.net
> >wrote:
>
> > You can lauch external processes from the couch config file.  One of
> those
> > could be a script that copies the database files to the target.
> >
> > If there are multiple machines and the databases must be merged, than a
> > collection of couch database servers on the target (central)  machine or
> > environment (on different ports, or running in virtual machines) can
> > replicate the docs into the main server... think of it like each VM is a
> > mirror of the remote database that is radio connected.
> >
> > Just my quick thoughts.
> >
> > -Scott
> >
> > "Sent from my T-Mobile 4G LTE Device", not some else's.
> >
> > -------- Original message --------
> > From: Aaron Huslage <huslage@gmail.com>
> > Date: 01/27/2014  10:30 AM  (GMT-06:00)
> > To: user@couchdb.apache.org
> > Subject: Replication without IP connection
> >
> > Hi,
> >
> > I have a strange problem that I'm hoping someone either has dealt with or
> > has ideas about solving.
> >
> > We have developed a medical records system as a CouchApp that allows
> > clinicians in remote areas to submit medical records to a local server.
> > These records need to be replicated back to a central office for
> reporting
> > purposes.
> >
> > These remote clinics are completely disconnected from the Internet and
> > connect to one another over HF Radio (think ham radio back in the day).
> The
> > radio links are not really reliable or quick enough for PPP to work well,
> > so we are using UUCP instead of IP.
> >
> > In this scenario, there is no interactive communication between nodes.
> > Everything is batch file copies. I need a way to do a CouchDB replication
> > using only file transfers and cron/inotify jobs.
> >
> > Any ideas or scripts would be helpful.
> >
> > Thanks,
> > Aaron
> >
> > --
> > Aaron Huslage
> > +1-919-600-1712
> > IM: GTalk - huslage@gmail.com; Skype - huslage
> >
> > PGP: 0x360DE590 / 30D4 B15B 74B2 F0E9 35D4  ABE0 2CB7 AC60 360D E590
> >
>
>
>
> --
> Aaron Huslage
> +1-919-600-1712
> IM: GTalk - huslage@gmail.com; Skype - huslage
>
> PGP: 0x360DE590 / 30D4 B15B 74B2 F0E9 35D4  ABE0 2CB7 AC60 360D E590
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message