Mailing-List: contact derby-dev-help@db.apache.org; run by ezmlm
Precedence: bulk
Reply-To: <derby-dev@db.apache.org>
Received-SPF: pass (asf.osuosl.org: domain of raymond_derby@hotmail.com
 designates 65.54.168.25 as permitted sender)
Message-ID: <BAY113-F15A1D177FBF221015719289A760@phx.gbl>
From: "Raymond Raymond" <raymond_derby@hotmail.com>
To: derby-dev@db.apache.org
Subject: A proposal on "how to determine if derby is busy"
Date: Tue, 25 Oct 2005 01:37:31 -0300
Mime-Version: 1.0
Content-Type: text/plain; format=flowed

Mike wrote:

Date:  Tue, 11 Oct 2005 10:57:08 -0700
>Are there any opinions out there on how to determine if
>Derby "is busy"?  Is there something better than just having
>a low priority thread  and maybe some query of cpu vs. elapsed
>time?

I have a proposal:
Since the disk IO is the bottleneck of the system performance,
we should consider it more than others, such as cpu usage.
Especially in the checkpointing issue, most of the workload of
checkpointing is to write out data to disk.So, instead of determining
if derby is busy, we can try to determine if derby IO is busy.
Is it possible to determine if derby IO is busy by figuring out how
many pages(cache pages) derby writes out to disk in every time
unit or how much time it takes derby to write out one page to
disk? I found derby writes out data to disk by pages. If we can
know how many pages derby writes out in every time unit, we
can determin if derby IO is busy in some extent. e.g. if derby
writes out lots of pages every second, we can say derby IO is busy.
If derby writes out a few pages every second, we can say derby IO
is not busy. In another way, we can dertermine if derby IO is busy
by figuring out how much time it takes derby to write one cache page.
If the time is long, it means the disk IO is busy. If the time is short,
it means the disk IO is not busy.

Does that sound resonable?Welcome everyone give your comment.


Raymond


>
>The first problem is that I don't think there are great tools
>for this in java.  The second problem is that often Derby is
>meant to be embedded as part of another application, so we have
>to be careful not to implement a standard server based approach
>where it is appropriate for the "server" to use up all resources
>available (ie. idle time may not really be best used by derby
>admin processes).
>
>I have not come up with a good answer to this problem, there are
>a number of things derby could do if it knew it had idle time
>available for it's use.  Best I have come up with is some mode
>in the system that needs to be set by the application which
>starts up Derby - either derby try's to limit it's use of idle
>cycles or it enabled to try and schedule work during idle time.
>
>Raymond Raymond wrote:
>
> > I have been thinking of the automatic checkpointing issue
> > recently.I also find someone added another issue about "Use
> > of idle time for background checkpoint" into the to-do list.
> > I think we can consider these two issue together. I have
> > some idea about it.
> >
> > Instead of doing checkpoint periodically and trying to tune the
> > checkpoint interval to achieve best performance, is it possible to
> > keep the background checkpoint process running to do checkpoint,
> > and the DBMS can tune the rate of checkpoint depending on the
> > current system situation,e.g. if the system is busy, derby will
> > slow down the checkpoint rate and if the system is not busy(idle),
> > derby will speed up the checkpoint rate.We will update the control
> > file periodically to let the DBMS know up to where we did checkpoint.
> > Maybe we can call it 'increamental checkpointing'. In my opinion,
> > this approach can use the disk IO resources with reason if we can
> > decide the checkpoint rate reasonablly.
> >
> > I would like to discuss this issue with everyone. I am not
> > sure if this approach is doable or not. If it is doable, I will
> > have some further questions about how to decide the appropriate
> > checkpoint rate.
> >
> > Thanks.
> >
> > Yours, Raymond
> >
> > _________________________________________________________________
> > Designer Mail isn't just fun to send, it's fun to receive. Use special
> > stationery, fonts and colors.
> > 
>http://join.msn.com/?pgmarket=en-ca&page=byoa/prem&xAPID=1994&DI=1034&SU=http://hotmail.com/enca&HL=Market_MSNIS_Taglines
> >  Start enjoying all the benefits of MSN?Premium right now and get the
> > first two months FREE*.
> >
> >

_________________________________________________________________
MSN� Calendar keeps you organized and takes the effort out of scheduling 
get-togethers. 
http://join.msn.com/?pgmarket=en-ca&page=byoa/prem&xAPID=1994&DI=1034&SU=http://hotmail.com/enca&HL=Market_MSNIS_Taglines 
  Start enjoying all the benefits of MSN� Premium right now and get the 
first two months FREE*.