db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jørgen Løland (JIRA) <j...@apache.org>
Subject [jira] Commented: (DERBY-2872) Add Replication functionality to Derby
Date Mon, 16 Jul 2007 11:34:06 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512921

Jørgen Løland commented on DERBY-2872:

>>* Derby doesn't log all operations by default, e.g. bulk
>> import, deleting all records from a table, creating an index. 
>"To make a consistent online backup in this scenario, this patch:
>1) blocks online backup until all the transactions with unlogged operation are
>    committed/aborted.
>2) implicitly converts all unlogged operations to logged mode for the duration
>    of the online backup, if they are started when backup is in progress. "

Thanks for investigating this Narayanan. Just to be sure: does
this mean that all operations are logged if we turn the
DERBY-239 mechanism on?

>>* Presumably, the master will time out if it is unable to send logs to
>> the slave for some (configurable?) period. It could keep trying for
>> some time but eventually it would need to stop or overflow the
>> buffer mechanism you suggest in DERBY-2926. Will you require that
>> the user has LOG_ARCHIVE_MODE enabled? If you do, it would seem a
>> nice addition to later be able to resume replication even if the
>> buffer had to be abandoned (as you suggest). *Before* the buffer is
>> abandoned, if the network becomes available again, it would be
>> trivial to resume shipping of logs I expect?

I think this functionality will have to be added as a later
extension to replication. When the issue is addressed, I do not
think we have to run with LOG_ARCHIVE_MODE enabled until a buffer
overflow occurs, however. When an overflow happens, the database should
be frozen, the existing replication buffer content written to
disk, and LOG_ARCHIVE_MODE be started. The database can then be 
unfrozen. This way, we do not need to store log files are written when 
replication works as it should.

I answered a related question in DERBY-2926; I'll just cut'n paste the
answer here for easy access:
>How do you imagine flow control if the network gets slow? Would you
>block a transaction whose record would overflow the buffer?

There are at least two simple alternatives for how to handle a
full replication buffer:

* Stop replication
* Block transactions

I think the first alternative would be the better one since
blocking transactions would mean no availability. This is the
exact opposite of what we want to achieve with replication.

The functional spec of DERBY-2872 states that resuming
replication after it has been stopped is a good candidate for
extending the functionality. Once that issue has been addressed,
we have a third alternative if the buffer gets full:

* Stop replication for now, but store the log files so that
  replication can be resumed later.

> Add Replication functionality to Derby
> --------------------------------------
>                 Key: DERBY-2872
>                 URL: https://issues.apache.org/jira/browse/DERBY-2872
>             Project: Derby
>          Issue Type: New Feature
>          Components: Miscellaneous
>    Affects Versions:
>            Reporter: Jørgen Løland
>            Assignee: Jørgen Løland
>         Attachments: proof_of_concept_master.diff, proof_of_concept_master.stat, proof_of_concept_slave.diff,
proof_of_concept_slave.stat, replication_funcspec.html, replication_funcspec_v2.html, replication_funcspec_v3.html,
> It would be nice to have replication functionality to Derby; many potential Derby users
seem to want this. The attached functional specification lists some initial thoughts for how
this feature may work.
> Dag Wanvik had a look at this functionality some months ago. He wrote a proof of concept
patch that enables replication by copying (using file system copy) and redoing the existing
Derby transaction log to the slave (unfortunately, I can not find the mail thread now).
> DERBY-2852 contains a patch that enables replication by sending dedicated logical log
records to the slave through a network connection and redoing these.
> Replication has been requested and discussed previously in multiple threads, including
> http://mail-archives.apache.org/mod_mbox/db-derby-user/200504.mbox/%3c426E04C1.1070904@yahoo.de%3e
> http://www.nabble.com/Does-Derby-support-Transaction-Logging---t2626667.html

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message