db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jørgen Løland (JIRA) <j...@apache.org>
Subject [jira] Commented: (DERBY-2926) Replication: Add a log buffer for log records that should be shipped to the slave
Date Fri, 13 Jul 2007 10:36:04 GMT

    [ https://issues.apache.org/jira/browse/DERBY-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12512423

Jørgen Løland commented on DERBY-2926:

Hi Mike

There is a better description of what we intend to create in DERBY-2872. Jira does not allow
multiple layers of subtasks, so I had to use the "is part of" link.

Basically, the goal is to provide asynchronous replication, i.e. replication where log shipment
to the slave is completely decoupled from transactions on the master. This is even looser
synchronization than your third alternative, which I believe is also known as 1-safe replication.

The asynchronous replication strategy may result in some lost transactions when the slave
performs fail-over. The amount of lost transactions is, of course, closely related to how
often log shipment is performed. As you mention, there is a trade off between how tight the
master/slave synchronization is and the incurred performance degradation. Since log shipping
in asynchronous replication is completely decoupled from the transactions, this strategy should
have less performance impact than the alternatives.

Although the plan is to add asynchronous replication now, replication with tighter synchronization
should be kept in mind. If possible within reasonable increased work, the architecture should
easily extend to 1-safe or 2-safe (your second alternative) replication later.

When it comes to the core replication functionality, there are two things the replication
master must know about: 1) log writes and 2) log flush. The log records must (sooner or later)
be sent to the slave, hence 1). What to do with flush calls is up to the replication strategy.
For the planned asynchronous replication, flush calls can be ignored. For 1-safe and 2-safe
replication, flush calls require log shipment as you describe in alternatives 2 and 3.

The current plan for using the log buffer is to append log records to it somewhere in LogToFile.appendLogRecord.
This is the same method used to append log records to logOut (output stream to the log file;
class type LogAccessFile). LogAccessFile is implemented with a number byte[] buffers (LogAccessFileBuffer),
which are ordered in a linked list. LogToFile is the only entry point for log writes, and
is therefore easily modifiable for our purpose. 

At the other end of the log buffer, a log shipping service will consume log records. The service
should, as you suggest, run as a separate thread. I think DaemonFactory could be useful to
create this thread, but that is just a guess. 

In the current code, the flush-methods in LogToFile are the only entry points for transactions
to force a log flush (e.g. at commit). Hence, adding forced log shipment to achieve 1 or 2
safe replication later can be easily put in these methods. In the planned asynchronous strategy,
log shipment may, e.g., be based on a timeout; flush calls can be ignored altogether.

I hope this clarifies most of your concerns. Does this architecture fit into your idea of
"tie into the existing log writing code" since log records are added to the buffer from the
log factory? The single entry point for both log writes and flushes makes this a good place
for modifications (for both asynchronous and the x-safe strategies) in my opinion, but there
may be good reasons for doing otherwise.

> Replication: Add a log buffer for log records that should be shipped to the slave
> ---------------------------------------------------------------------------------
>                 Key: DERBY-2926
>                 URL: https://issues.apache.org/jira/browse/DERBY-2926
>             Project: Derby
>          Issue Type: Sub-task
>          Components: Store
>    Affects Versions:
>            Reporter: Jørgen Løland
>            Assignee: Jørgen Løland
> When a Derby instance has the master role for a database, log records are shipped to
the slave to keep it up to date. A buffer is needed because the log records should not be
shipped one at a time. Also, writing the log records to a buffer instead of sending them immediately
removes the network communication from the critical path for the transaction.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message