Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Message-ID: <23619919.1185276571442.JavaMail.jira@brutus>
Date: Tue, 24 Jul 2007 04:29:31 -0700 (PDT)
From: =?utf-8?Q?J=C3=B8rgen_L=C3=B8land_=28JIRA=29?= <jira@apache.org>
To: derby-dev@db.apache.org
Subject: [jira] Commented: (DERBY-2872) Add Replication functionality to
 Derby
In-Reply-To: <16039538.1182851726399.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/DERBY-2872?page=3Dcom.atlassian=
.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514942 ]=20

J=C3=B8rgen L=C3=B8land commented on DERBY-2872:
--------------------------------------

>> From the adminguide, topic "Online backups":
> <snip>
>
>This citation is from the 10.1 version of the guide.  If you look at
>the 10.2 guide, you will see that it has been changed and that the
>backup is now non-blocking.

Actually, that was from the 10.2 documentation. I also checked the alpha ma=
nuals, and could not find anything about non-blocking. However, that does n=
ot change the fact that backup IS performed non-blocking (searching the web=
 revealed a few presentations stating so).=20

Since I was not previously aware of non-blocking backup, some adjustments w=
ill probably be made to our development plan and the funcspec. More precise=
ly, it would be great if non-blocking online backup would provide us the to=
ol to avoid freezing the database. I also agree that it should not be neces=
sary to freeze the database only to start at the correct log instant 'i'.

>Another issue to think about:
>
>If an transaction has been open since before replication was started, a
>slave may need log from before log shipping started to be able to undo
>the transaction at failover time.  In order to be sure that failover
>will succeed, I think there are two alternatives:
> 1) Let the master abort/rollback such transactions before it
>    acknowledges that replication has started.
> 2) Send old log records to the slave to make sure it has all
>    necessary log to do failover.
>
>I think the approach of non-blocking backup is similar to alternative
>2). That is, it copies all log files that may be necessary in order to
>perform undo at restore.

I have not had the time to figure out how non-blocking backup works, but a =
shot in the dark would be that it works in a similar way as fuzzy checkpoin=
ting. As you say, the non-blocking backup mechanism has to ensure a solutio=
n to this problem. Using the same solution is probably a good thing.

With this new information, I think the revised plan will be something like =
this:

1) A Derby is informed that it will become master for db 'x'.=20
2) The master starts making an online, non-blocking backup. The location of=
 this backup may be disk or even sent over the network to the slave. This h=
as not been decided.
3) Online backup completes. From this point on, all operations are logged b=
oth to disk and to the replication buffer.=20
4) If the backup location was disk, the entire backup is sent to the slave.=
 The disk-version can then be deleted.
5) When the backup has been sent to the slave, the master starts sending lo=
g from the log buffer. Once the slave has received enough log to be "up to =
date" (depends on how tightly synchronized the master and slave is), replia=
tion is reported to have started. Since we are currently working on asynchr=
onous replication, we can report that replication has started when we start=
 shipping buffered log records.

Note that this does not require any database freeze as long as backup can b=
e performed without freeze.

Pros and cons versus the previous plan:
+ Non-blocking
- Requires 2x diskspace during steps 2-4
- Requires more memory from step 3 to the point where the slave has caught =
up with the master.

> Add Replication functionality to Derby
> --------------------------------------
>
>                 Key: DERBY-2872
>                 URL: https://issues.apache.org/jira/browse/DERBY-2872
>             Project: Derby
>          Issue Type: New Feature
>          Components: Miscellaneous
>    Affects Versions: 10.4.0.0
>            Reporter: J=C3=B8rgen L=C3=B8land
>            Assignee: J=C3=B8rgen L=C3=B8land
>         Attachments: proof_of_concept_master.diff, proof_of_concept_maste=
r.stat, proof_of_concept_slave.diff, proof_of_concept_slave.stat, replicati=
on_funcspec.html, replication_funcspec_v2.html, replication_funcspec_v3.htm=
l, replication_script.txt
>
>
> It would be nice to have replication functionality to Derby; many potenti=
al Derby users seem to want this. The attached functional specification lis=
ts some initial thoughts for how this feature may work.
> Dag Wanvik had a look at this functionality some months ago. He wrote a p=
roof of concept patch that enables replication by copying (using file syste=
m copy) and redoing the existing Derby transaction log to the slave (unfort=
unately, I can not find the mail thread now).
> DERBY-2852 contains a patch that enables replication by sending dedicated=
 logical log records to the slave through a network connection and redoing =
these.
> Replication has been requested and discussed previously in multiple threa=
ds, including these:
> http://mail-archives.apache.org/mod_mbox/db-derby-user/200504.mbox/%3c426=
E04C1.1070904@yahoo.de%3e
> http://www.nabble.com/Does-Derby-support-Transaction-Logging---t2626667.h=
tml

--=20
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.