db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "V.Narayanan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-3359) The log shipper needs to be modified to vary the shipping intervals dynamically (based on load)
Date Thu, 31 Jan 2008 10:03:34 GMT

    [ https://issues.apache.org/jira/browse/DERBY-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564353#action_12564353
] 

V.Narayanan commented on DERBY-3359:
------------------------------------

Implementation Nuances
----------------------

Here, I will be explaining the changes that will be required to the appropriate classes for
implementating the Dynamic log shipping functionality as detailed above.

org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer
----------------------------------------------------------------------

This class will contain the functionality for 

1) calculating the Fill Information (*FI*)
2) For notifying the log shipper each time a *log buffer element* becomes full.

1) calculating the Fill Information (*FI*)
------------------------------------------

FI will be an integer with a value ranging between 0-100

FI = (dirtyBuffers.size()/DEFAULT_NUMBER_LOG_BUFFERS)*100

where,

dirtyBuffers is a LinkedList containing the log buffers elements with unsent log.

DEFAULT_NUMBER_LOG_BUFFERS The total number of log buffers elements (filled and unfilled).

A method,

public int getFillInformation();

will be added that will return the value of FI as calculated above.

2) For notifying the log shipper each time a log buffer element becomes full.
----------------------------------------------------------------------------

Each time a log shipper element becomes full we call *switchDirtyBuffer()* in order to
Append the currentDirtyBuffer to dirtyBuffers, and make a fresh buffer element from 
freeBuffers.

We will add logic in the switchDirtyBuffer() method for notifying the Log Shipper.

For more information on the LogShipper please refer to,
1) http://issues.apache.org/jira/browse/DERBY-2926
2) http://issues.apache.org/jira/browse/DERBY-3051


org.apache.derby.impl.services.replication.master.LogShipper (interface)
------------------------------------------------------------------------

This interface will contain the definition of the method that will be called by the
log buffer.

Will add a method workToDo that will be used by the Log Buffer to inform the log shipper
whenever a Log Buffer Element becomes full.

org.apache.derby.impl.services.replication.master.AsynchronousLogShipper
------------------------------------------------------------------------

This class will contain the functionality for managing load in the Log buffer (by shipping
log records)
,by using the FI information, during each buffer element full notification that is received.

Two constants
-------------

MID = 50ms
MAX = 550ms (MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID)

attribute nextShippingTime
--------------------------

will store the time at which the next log ship will happen and will be calculated as System.currentTimeMillis()
+
shippingInterval in the run method.

methods
-------

constructor
-----------

shippingInterval will be initialized to MID in the beginning.

nextShippingTime will be initialized in the constructor initially to 
System.currentTimeMills() + shippingInterval (i.e. MID).


public void workToDo();
-----------------------

Will be added that will be called by the log buffer each time a buffer element becomes full.

The following steps will be used in this method

a) Get FI from log buffer
b) If FI >= 80
   b.1) call shipALogChunk()
   b.2) repeat b) until FI < 80
   b.3) notify the log shipper thread.
c) If FI > 10 && FI < 80
   c.1) calculate the time elapsed since last log ship.
        (For this before calling wait we need to store the
        time at which next transmit will happen.)
   c.2) If the time remaining for the next ship is less than
        MID then do not do anything.
   c.3) If the time elapsed since last ship is greater than
        MID notify the log shipper thread. (here we could have
        simply not shipped and waited for timeout to happen, but
        this would be a risk and a better option would be to ship)

will check if the forceFlush method is required at all since all of the work done by the forceFlush
method can now be done by the workToDo method (see changes to MasterController).

public void calculateWTfromFI() (WT - wait time)
-------------------------------

will be used to calculate the wait time using the FI from the buffer.

shippingInterval will be assigned to the return value from this method before calling
wait(shippingInterval) in the run method. *If this method returns -1 however we should
not call wait but just continue with the shipping.*

The following steps will be used in this method

a) FI >= 80 return -1 (*we should not call wait if the return value is -1*)
b) FI > 10 and FI < 80 return MID
c) FI <= 10 return MAX

run()
-----

This method will be modified to

1) shippingInterval will be assigned the return value
   of calculateWTfromFI().
2) calculate the next shipping time  which will be 
   System.currentTimeMillis() + shippingInterval.

For more information on the log shipper please refer to
http://issues.apache.org/jira/browse/DERBY-3064

org.apache.derby.impl.services.replication.master.MasterController
------------------------------------------------------------------

When a LogBufferFullException is thrown we will call workToDo instead of forceFlush.

For more information on the MasterController please refer to
http://issues.apache.org/jira/browse/DERBY-2977

> The log shipper needs to be modified to vary the shipping intervals dynamically (based
on load)
> -----------------------------------------------------------------------------------------------
>
>                 Key: DERBY-3359
>                 URL: https://issues.apache.org/jira/browse/DERBY-3359
>             Project: Derby
>          Issue Type: Sub-task
>            Reporter: V.Narayanan
>            Assignee: V.Narayanan
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message