db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Matrigali <mikem_...@sbcglobal.net>
Subject Re: Replication
Date Fri, 25 Jul 2008 21:48:10 GMT
V Narayanan wrote:
> Hi Mike,
> Currently the replication functionality is not a pluggable module but is 
> rather strongly linked with the store
> layer. For example LogToFile has been actually modified to catch the log 
> records and store them in the log
> buffer in addition to storing them on the master. If you are thinking 
> about implementing a synchronous or
> a synchronous (but read-only masters, I think in this case also you 
> would be constrained to commit a transaction
> only after all the masters sync), you would do great to think of how 
> this can be implemented as a separate module
> rather than coupling it tightly with the store as is happening now.
> You could also think about convincing the community about the advantages 
> a synchronous replication scheme might
> bring to Derby over the existing asynchronous scheme and probably think 
> about modifying existing implementation
> to work in a sychronous way, or atleast think about how the existing 
> functionality can be bundled as a separate unit.
I would also suggest looking at adding a new replication module rather
than change the existing code.  The system can be set up such that the
new replication module can extend the existing functionality to achieve
the additional work necessary.  It was how I had expected a hot standby
to be implemented.

I think a read only replicant could be implemented using a lot of 
existing functionality.  With
changes that did:
1) add replication module(s)
2) module to extend log buffer writes in some way such that log records 
are sent to read only replicant some how.  How and how often depend on
how current you need the read only replicant.  You could probaby even
do this with no in line changes to log writing and just read the log
files from disk along with assuring the db is in backup mode which 
always keeps all log files.  This mode would be interesting as it would
inpact the master performance as little as possible (except for possible
I/O throughput to log device).
3) add new module that applies log records and thus changes to the db, 
but does not allow any external changes.  To support read committed or 
higher than some extra work may need to be done in this area to get 
expected locks while applying log records.  Currently the code that does
this assumes it is boot time and need not get locks.  Basically this
code wants to be a copy of the code that is done during boot recovery
but handle applying the log records "online" vs. "offline".  This is
probably the most challenging bit.

> Some good places to start would be
> http://issues.apache.org/jira/browse/Derby-2872
> http://wiki.apache.org/db-derby/ReplicationWriteup
> You could also look at some synchronous replication products listed here
> http://wiki.apache.org/db-derby/UsesOfDerby
> looking at them would probably help you what type of an end product you 
> would expect.
> There was an effort from a student to provide hot standby functionality 
> to derby, you can find the paper and related work here
> https://issues.apache.org/jira/browse/DERBY-2852
> I will help you in whatever way I can in this effort, but I may not 
> actually be operation in synchronous mode ;) , I promise
> asynchronous read-only, commit-maybe, support to your effort :) .
> But your proposal looks interesting, please do remember to keep the 
> community informed about your ideas, efforts and problems
> constantly. I have noticed that the community appreciates continuous 
> feedback on issues.
> Narayanan
> Mike Boom wrote:
>> I may want to add master/master replication or at least master/readonly
>> replication.  How would I find out who the original contributors would
>> be?
>> -----Original Message-----
>> From: Richard.Hillegas@Sun.COM [mailto:Richard.Hillegas@Sun.COM] 
>> Sent: Monday, July 21, 2008 12:42 PM
>> To: derby-dev@db.apache.org
>> Subject: Re: Replication
>> Mike Boom wrote:
>>> I hope this is an appropriate place to ask this question. Is there a 
>>> plan in the future for replication to provide readable replicas, or 
>>> even better multiple masters?  If so what kind of time line are we 
>>> looking at for completion?  Thanks.
>> Hi Mike,
>> I have not seen a lot of work on replication recently, other than 
>> stabilization of the existing functionality and tests. Probably, the 
>> original contributors would be happy to coach someone who wanted to 
>> extend Derby replication.
>> Regards,
>> -Rick

View raw message