db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "V.Narayanan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (DERBY-3064) Implement the LogShipper that will enable the shipping of Log records from the master to the slave
Date Mon, 24 Sep 2007 09:31:50 GMT

    [ https://issues.apache.org/jira/browse/DERBY-3064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12529813

V.Narayanan commented on DERBY-3064:

>1. If I understand you correctly, log records will be shipped either
>   when the log buffer is full or when a timeout occur. If the traffic
>   is high so that the time it takes to fill the log buffer is
>   normally less than the timeout, this means that log will only be
>   sent when the log buffer is full. Would it not be better to try to
>   keep the log shipping ahead so that it never goes full?

What we can do is we start with a default value for transmit interval.
For theory sake let us assume it is 3 seconds.

Suppose the condition stated on your comments arise, we maintain a counter
that keeps track of the number of times force flush is called, if this counter
value exceeds a certain value we can replace the transmit interval with
the last transmit interval we obtained.

Thus this would automatically take care of the case of heavy load.

A similar check should run in performWork to take care of high load.

Both these counters should be reset when the other method is called.

The counter for forceFlush is reset when performWork is called and vice-versa.

I however do not want to do this in the first version of the patch for this issue. 
I will submit the first version assuming a static time interval, keep this issue
open and change it in the next version if this is OK with the community.

I feel this will put in the essential features first and the improvements later.

>2. It seems like you suggest that the shipping thread will continously
>   check to see whether the timeout has expired. Generally, such
>   busy-waits should be avoided. I think you should introduce a timer
>   or something so that the thread can be suspended until it is time
>   to ship log records. 

I have introduced the time concept to overcome the limitation imposed by

The DaemonFactory would call performWork (i.e.) ship log records at its own
convinience. As i understand it we cannot configure DaemonFactory to transmit
at our mentioned time interval.

So, When performWork is called entirely depends on how good DaemonFactory is feeling :-)

We do not want this to happen, because what if this is too soon for us or what if it
is too late.

Hence we introduce a transmit interval variable that calculates the difference between the
last call and the current call of performWork or rather shipALogChunk(forceFlush also)

>3. You mention a ShippingDeamon, but it is not described what it is or
>   how it works. Is this a new thread that you will introduce or will
>   the LogShipper be run in the existing background thread? 

Sorry about being ambiguous here. ShippingDaemon is Derby's DaemonService that is booted
up and is not a thread I introduce.

> Implement the LogShipper that will enable the shipping of Log records from the master
to the slave
> --------------------------------------------------------------------------------------------------
>                 Key: DERBY-3064
>                 URL: https://issues.apache.org/jira/browse/DERBY-3064
>             Project: Derby
>          Issue Type: Sub-task
>            Reporter: V.Narayanan
>            Assignee: V.Narayanan
>         Attachments: LogShipperImpl_v1.diff, LogShipperImpl_v1.stat

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message