Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Message-ID: <31337842.1201602334259.JavaMail.jira@brutus>
Date: Tue, 29 Jan 2008 02:25:34 -0800 (PST)
From: "V.Narayanan (JIRA)" <jira@apache.org>
To: derby-dev@db.apache.org
Subject: [jira] Issue Comment Edited: (DERBY-3359) The log shipper needs to
 be modified to vary the shipping intervals dynamically (based on load)
In-Reply-To: <27962061.1201602094081.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/DERBY-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563453#action_12563453 ] 

narayanan edited comment on DERBY-3359 at 1/29/08 2:24 AM:
-------------------------------------------------------------

Introduction
------------

The log shipper currently has a static shipping interval of 1 second with
a forceFlush capability when the buffer becomes full. This algorithm does not
augur well in the real-time environment and the bond between the buffer fill
and the log shipper frequency needs to be strengthened to allow shipping
to depend on incoming transaction load.

Keeping this in mind the following refined shipping scheme is proposed for the
first version of replication. If required the design can be later modified to
a more efficient and complex implementations in later refinements of replication.


Basic Idea
----------

The Log Buffer will notify the Log Shipper when a buffer is full. When the Log 
Shipper gets this notification it will decide the action to be done the following
way

1) It will retrieve the fill information from the Log Buffer. The fill information
   is a indicator of how full the log is at any point of time.

   fill information = (full buffers/Total Buffers)*100

2) If the fill information is greater than 80 the log shipper will ship log records
   until the fill information returns a value less than 80. There will be no time
   delay between the ships.

3) If the fill information is less than 80 but greater than 10 the log shipper will
   ship with a MID ms delay. (*MID has not been decided yet*)

4) If the fill information is less than 10 the log shipper will ship with a MAX ms delay
   or when a buffer becomes full whichever comes first. The delay however will not be
   greater than MID.
   (*MAX > MID exact values of MAX has not been decided yet*).

Note: MID is a value that is only as large as not to affect the performance of the master database
      significantly.

      max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MID)  is the maximum delay between a log record is committed at the master until 
      it is replicated to the slave. Maybe we should make sure that MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID. 

      was (Author: narayanan):
    Keeping this in mind the following refined shipping scheme is proposed for the
first version of replication. If required the design can be later modified to
a more efficient and complex implementations in later refinements of replication.


Basic Idea
----------

The Log Buffer will notify the Log Shipper when a buffer is full. When the Log 
Shipper gets this notification it will decide the action to be done the following
way

1) It will retrieve the fill information from the Log Buffer. The fill information
   is a indicator of how full the log is at any point of time.

   fill information = (full buffers/Total Buffers)*100

2) If the fill information is greater than 80 the log shipper will ship log records
   until the fill information returns a value less than 80. There will be no time
   delay between the ships.

3) If the fill information is less than 80 but greater than 10 the log shipper will
   ship with a MID ms delay. (*MID has not been decided yet*)

4) If the fill information is less than 10 the log shipper will ship with a MAX ms delay
   or when a buffer becomes full whichever comes first. The delay however will not be
   greater than MID.
   (*MAX > MID exact values of MAX has not been decided yet*).

Note: MID is a value that is only as large as not to affect the performance of the master database
      significantly.

      max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MID)  is the maximum delay between a log record is committed at the master until 
      it is replicated to the slave. Maybe we should make sure that MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID. 
  
> The log shipper needs to be modified to vary the shipping intervals dynamically (based on load)
> -----------------------------------------------------------------------------------------------
>
>                 Key: DERBY-3359
>                 URL: https://issues.apache.org/jira/browse/DERBY-3359
>             Project: Derby
>          Issue Type: Sub-task
>            Reporter: V.Narayanan
>            Assignee: V.Narayanan
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.