db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jørgen Løland (JIRA) <j...@apache.org>
Subject [jira] Commented: (DERBY-2872) Add Replication functionality to Derby
Date Fri, 22 Feb 2008 08:28:19 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571306#action_12571306

Jørgen Løland commented on DERBY-2872:

Thanks for reviewing the funcspec. I'll answer your questions inline. 

> 1. Would be nice to also describe a scenario where the slave Derby instance crashes

I agree - I'll add the following information in the next rev of the funcspec:

If the slave crashes, the master will try to reestablish the connection (but will not make
it) until the replication log buffer is full or until the master is told to stop replication

> 2. Please confirm the following statement: 'Since the slave database is in recovery-mode,
no data in the slave database can be changed at all by any means'. If this wouldn't be the
case, a lot of consistency-problems and scenarios should be taken into account.

Confirmed. We do not allow any operation on the slave instance of the database, not even reads.
(Although reads may me allowed in a later (post 10.4) improvement.)

> 3. Is the relation Master-Slave really an N:M-relation and if so, are N and/or M somehow

1:1 - There will be exactly one master and one slave for each replicated database 'x'. However,
one Derby instance may act as a master or a slave for many different databases. 

In the following example, there are three Derby instances managing a total of five databases.
The databases are either booted the normal non-replicated way, in master mode or in slave
mode. There is exactly one master and one slave per replicated db:

Derby Inst 1      Derby Inst 2      Derby Inst 3
Normal: 'db1'  
Master: 'db2'     Slave:  'db2'
Master: 'db3'                                Slave:  'db3'
Slave:  'db4'       Master: 'db4'
                                                       Normal: 'db5'

> 4. Master has still connection with Slave, but for some reason the Slave does not process
the received logrecords: will this be seen as 'unexpected failure'?

Yes. That would be a code bug, and considered failsafe because the slave code checks that
each and every log record it applies to the local log has the exact same byte-position as
it had on the master side. If a single log record is lost somewhere, the slave will notice.
Of course, there could be other bugs like the slave acting as a black hole (receiving packages
and replying with an ack (implicitly through TCP), but not adding the received log records
to the local log). That kind of bug would not be noticed until failover

> 5. Network issue: what to do in case of missing packages?
> 6. Network issue: will a multipath-configuration be supported? This will lead to a Slave
receiving records in a wrong sequence
> 7. Network issue: multiple-network-interfaces: question as for 6). In addition: special
scenario(s) if one such interface is dropped?

The network communication is based on a TCP on one single socket. The connection ensures correct
ordering of packages since the TCP protocol confirms that each package is delivered. A new
package is not sent before the previous has been acked.

> 8. If 1 Master replicates to 2 different Slaves and one of these Slaves has a lower 'bandwidth'
to receive/process the logs, will this one then determine the total (replication)transactional
processes, thus also limit sending to the other Slave?

The 'one master, one slave' configuration prevents this.

> 9. If Derby instance I1 contains database D1 as Master, database D2 as Slave, instance
I2 contains the same but with opposite roles: isn't there a big risk for a 'dead' race in
case of problems with main memory log buffer?

The replication log buffers are not shared. Each master will have it's own main memory replication
log buffer. (Could you please rephrase the question if this answer was gibberish?)

> Add Replication functionality to Derby
> --------------------------------------
>                 Key: DERBY-2872
>                 URL: https://issues.apache.org/jira/browse/DERBY-2872
>             Project: Derby
>          Issue Type: New Feature
>          Components: Replication
>    Affects Versions:
>            Reporter: Jørgen Løland
>            Assignee: Jørgen Løland
>         Attachments: master_classes_1.pdf, poc_master_v2.diff, poc_master_v2.stat, poc_master_v2b.diff,
poc_slave_v2.diff, poc_slave_v2.stat, poc_slave_v2b.diff, poc_slave_v2c.diff, proof-of-concept_v2b-howto.txt,
proof_of_concept_master.diff, proof_of_concept_master.stat, proof_of_concept_slave.diff, proof_of_concept_slave.stat,
replication_funcspec.html, replication_funcspec_v2.html, replication_funcspec_v3.html, replication_funcspec_v4.html,
replication_funcspec_v5.html, replication_funcspec_v6.html, replication_funcspec_v7.html,
replication_funcspec_v8.html, replication_funcspec_v9.html, replication_script.txt, slave_classes_1.pdf
> It would be nice to have replication functionality to Derby; many potential Derby users
seem to want this. The attached functional specification lists some initial thoughts for how
this feature may work.
> Dag Wanvik had a look at this functionality some months ago. He wrote a proof of concept
patch that enables replication by copying (using file system copy) and redoing the existing
Derby transaction log to the slave (unfortunately, I can not find the mail thread now).
> DERBY-2852 contains a patch that enables replication by sending dedicated logical log
records to the slave through a network connection and redoing these.
> Replication has been requested and discussed previously in multiple threads, including
> http://mail-archives.apache.org/mod_mbox/db-derby-user/200504.mbox/%3c426E04C1.1070904@yahoo.de%3e
> http://www.nabble.com/Does-Derby-support-Transaction-Logging---t2626667.html

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message