db-derby-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "David W. Van Couvering" <David.Vancouver...@Sun.COM>
Subject Re: A proposal on "how to determine if derby is busy"
Date Tue, 25 Oct 2005 15:25:14 GMT
Hm, an interesting idea, but methinks what makes Derby "busy" depends on 
your configuration.  If you have made a big enough page cache that the 
entire DB is in memory, perhaps CPU is the bottleneck.  Or if you are 
storing on RAM disk, or if you have the db durability set to "test" -- 
in other words, I wonder if not all configurations are made equal in 
terms of what makes Derby busy.

Also, CPU usage is a big issue in client/server mode -- the network 
server could be busy simply trying to handle all the network traffic, 
and that could be the bottleneck for the system if there is a high 
client load...

David

Raymond Raymond wrote:
> Mike wrote:
> 
> Date:  Tue, 11 Oct 2005 10:57:08 -0700
> 
>> Are there any opinions out there on how to determine if
>> Derby "is busy"?  Is there something better than just having
>> a low priority thread  and maybe some query of cpu vs. elapsed
>> time?
> 
> 
> I have a proposal:
> Since the disk IO is the bottleneck of the system performance,
> we should consider it more than others, such as cpu usage.
> Especially in the checkpointing issue, most of the workload of
> checkpointing is to write out data to disk.So, instead of determining
> if derby is busy, we can try to determine if derby IO is busy.
> Is it possible to determine if derby IO is busy by figuring out how
> many pages(cache pages) derby writes out to disk in every time
> unit or how much time it takes derby to write out one page to
> disk? I found derby writes out data to disk by pages. If we can
> know how many pages derby writes out in every time unit, we
> can determin if derby IO is busy in some extent. e.g. if derby
> writes out lots of pages every second, we can say derby IO is busy.
> If derby writes out a few pages every second, we can say derby IO
> is not busy. In another way, we can dertermine if derby IO is busy
> by figuring out how much time it takes derby to write one cache page.
> If the time is long, it means the disk IO is busy. If the time is short,
> it means the disk IO is not busy.
> 
> Does that sound resonable?Welcome everyone give your comment.
> 
> 
> Raymond
> 
> 
> 
> 
>>
>> The first problem is that I don't think there are great tools
>> for this in java.  The second problem is that often Derby is
>> meant to be embedded as part of another application, so we have
>> to be careful not to implement a standard server based approach
>> where it is appropriate for the "server" to use up all resources
>> available (ie. idle time may not really be best used by derby
>> admin processes).
>>
>> I have not come up with a good answer to this problem, there are
>> a number of things derby could do if it knew it had idle time
>> available for it's use.  Best I have come up with is some mode
>> in the system that needs to be set by the application which
>> starts up Derby - either derby try's to limit it's use of idle
>> cycles or it enabled to try and schedule work during idle time.
>>
>> Raymond Raymond wrote:
>>
>> > I have been thinking of the automatic checkpointing issue
>> > recently.I also find someone added another issue about "Use
>> > of idle time for background checkpoint" into the to-do list.
>> > I think we can consider these two issue together. I have
>> > some idea about it.
>> >
>> > Instead of doing checkpoint periodically and trying to tune the
>> > checkpoint interval to achieve best performance, is it possible to
>> > keep the background checkpoint process running to do checkpoint,
>> > and the DBMS can tune the rate of checkpoint depending on the
>> > current system situation,e.g. if the system is busy, derby will
>> > slow down the checkpoint rate and if the system is not busy(idle),
>> > derby will speed up the checkpoint rate.We will update the control
>> > file periodically to let the DBMS know up to where we did checkpoint.
>> > Maybe we can call it 'increamental checkpointing'. In my opinion,
>> > this approach can use the disk IO resources with reason if we can
>> > decide the checkpoint rate reasonablly.
>> >
>> > I would like to discuss this issue with everyone. I am not
>> > sure if this approach is doable or not. If it is doable, I will
>> > have some further questions about how to decide the appropriate
>> > checkpoint rate.
>> >
>> > Thanks.
>> >
>> > Yours, Raymond
>> >
>> > _________________________________________________________________
>> > Designer Mail isn't just fun to send, it's fun to receive. Use special
>> > stationery, fonts and colors.
>> > 
>> http://join.msn.com/?pgmarket=en-ca&page=byoa/prem&xAPID=1994&DI=1034&SU=http://hotmail.com/enca&HL=Market_MSNIS_Taglines

>>
>> >  Start enjoying all the benefits of MSN?Premium right now and get the
>> > first two months FREE*.
>> >
>> >
> 
> 
> _________________________________________________________________
> MSNĀ® Calendar keeps you organized and takes the effort out of scheduling 
> get-togethers. 
> http://join.msn.com/?pgmarket=en-ca&page=byoa/prem&xAPID=1994&DI=1034&SU=http://hotmail.com/enca&HL=Market_MSNIS_Taglines

>  Start enjoying all the benefits of MSNĀ® Premium right now and get the 
> first two months FREE*.
> 

Mime
View raw message