db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jørgen Løland (JIRA) <j...@apache.org>
Subject [jira] Commented: (DERBY-2872) Add Replication functionality to Derby
Date Tue, 24 Jul 2007 11:29:31 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12514942

Jørgen Løland commented on DERBY-2872:

>> From the adminguide, topic "Online backups":
> <snip>
>This citation is from the 10.1 version of the guide.  If you look at
>the 10.2 guide, you will see that it has been changed and that the
>backup is now non-blocking.

Actually, that was from the 10.2 documentation. I also checked the alpha manuals, and could
not find anything about non-blocking. However, that does not change the fact that backup IS
performed non-blocking (searching the web revealed a few presentations stating so). 

Since I was not previously aware of non-blocking backup, some adjustments will probably be
made to our development plan and the funcspec. More precisely, it would be great if non-blocking
online backup would provide us the tool to avoid freezing the database. I also agree that
it should not be necessary to freeze the database only to start at the correct log instant

>Another issue to think about:
>If an transaction has been open since before replication was started, a
>slave may need log from before log shipping started to be able to undo
>the transaction at failover time.  In order to be sure that failover
>will succeed, I think there are two alternatives:
> 1) Let the master abort/rollback such transactions before it
>    acknowledges that replication has started.
> 2) Send old log records to the slave to make sure it has all
>    necessary log to do failover.
>I think the approach of non-blocking backup is similar to alternative
>2). That is, it copies all log files that may be necessary in order to
>perform undo at restore.

I have not had the time to figure out how non-blocking backup works, but a shot in the dark
would be that it works in a similar way as fuzzy checkpointing. As you say, the non-blocking
backup mechanism has to ensure a solution to this problem. Using the same solution is probably
a good thing.

With this new information, I think the revised plan will be something like this:

1) A Derby is informed that it will become master for db 'x'. 
2) The master starts making an online, non-blocking backup. The location of this backup may
be disk or even sent over the network to the slave. This has not been decided.
3) Online backup completes. From this point on, all operations are logged both to disk and
to the replication buffer. 
4) If the backup location was disk, the entire backup is sent to the slave. The disk-version
can then be deleted.
5) When the backup has been sent to the slave, the master starts sending log from the log
buffer. Once the slave has received enough log to be "up to date" (depends on how tightly
synchronized the master and slave is), repliation is reported to have started. Since we are
currently working on asynchronous replication, we can report that replication has started
when we start shipping buffered log records.

Note that this does not require any database freeze as long as backup can be performed without

Pros and cons versus the previous plan:
+ Non-blocking
- Requires 2x diskspace during steps 2-4
- Requires more memory from step 3 to the point where the slave has caught up with the master.

> Add Replication functionality to Derby
> --------------------------------------
>                 Key: DERBY-2872
>                 URL: https://issues.apache.org/jira/browse/DERBY-2872
>             Project: Derby
>          Issue Type: New Feature
>          Components: Miscellaneous
>    Affects Versions:
>            Reporter: Jørgen Løland
>            Assignee: Jørgen Løland
>         Attachments: proof_of_concept_master.diff, proof_of_concept_master.stat, proof_of_concept_slave.diff,
proof_of_concept_slave.stat, replication_funcspec.html, replication_funcspec_v2.html, replication_funcspec_v3.html,
> It would be nice to have replication functionality to Derby; many potential Derby users
seem to want this. The attached functional specification lists some initial thoughts for how
this feature may work.
> Dag Wanvik had a look at this functionality some months ago. He wrote a proof of concept
patch that enables replication by copying (using file system copy) and redoing the existing
Derby transaction log to the slave (unfortunately, I can not find the mail thread now).
> DERBY-2852 contains a patch that enables replication by sending dedicated logical log
records to the slave through a network connection and redoing these.
> Replication has been requested and discussed previously in multiple threads, including
> http://mail-archives.apache.org/mod_mbox/db-derby-user/200504.mbox/%3c426E04C1.1070904@yahoo.de%3e
> http://www.nabble.com/Does-Derby-support-Transaction-Logging---t2626667.html

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message