[ https://issues.apache.org/jira/browse/DERBY-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046365#comment-13046365 ] 

Brett Wooldridge commented on DERBY-5265:
-----------------------------------------

I suspected that was they reason for the current in-memory implementation.

I actually have been thinking quite a lot about replication, and a proposal is forming in my head for a fully automated failover/failback, including two-way replication and transaction log replication markers.  As noted, log deletion/checkpointing code would definitely be affected under such a scenario.

Where is a good place to start to document such a proposal?  Is there a development wiki available?


> LOB replication limit
> ---------------------
>
>                 Key: DERBY-5265
>                 URL: https://issues.apache.org/jira/browse/DERBY-5265
>             Project: Derby
>          Issue Type: Improvement
>          Components: Replication
>            Reporter: Brett Wooldridge
>
> I have an issue with replication as it pertains to LOBs (BLOBs and CLOBs).
> According to the documentation...
> If the master looses connection with the slave, "transactions are
> allowed to continue processing while the master tries to reconnect with 
> the slave. Log records generated while the connection is down are 
> buffered in main memory. If the log buffer reaches its size limit before 
> the connection can be reestablished, the master replication
> functionality is stopped."
> And the documentation for derby.replication.logBufferSize says the
> maximum size of the buffer is 1048576 (1MB).
> This seems to imply that if I have a database in which I store LOBs
> which are, for example, 256K in size, and the connection between 
> master and slave is severed, I can perform 4 inserts or less before 
> the master gives up.  I would like to file a request that this limit be raised
> considerably or eliminated altogether.
> I have two servers (master and slave) running 64-bit JVMs, 64GB of memory each,
> SSD drives, connected by 10GbE fiber.  I would like to dedicate as much memory 
> as I want to deal with a disconnect/resume scenario (to avoid the onerous failover).  
> At an insertion rate of 16 rows per second (~4MB), currently the setup would
> tolerate a connection interruption of a fraction of a second.  A 1GB buffer would
> afford a connection interruption of ~250 seconds (for example, rebooting the fiber 
> switch).
> Lastly, why does Derby even bother to buffer logs in memory?  Can't it just keep 
> an offset/marker into the transaction log files, or better insert a special replication
> log entry, and replay transactions from there, rather than buffering them in memory?

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira